DSL (Domain Specific Language)

A Domain-Specific Language (DSL) is a computer language targeted at a particular kind of problem, rather than a general-purpose language aimed at any kind of software problem.

Domain-specific languages have been discussed and used for almost as long as computing has existed. In the data space, they are mostly used with YAML. For example, Dagster uses a DSL to declaratively create a pipeline/graph ( docs): ^8853ef

Sometimes, you may want to construct the dependencies of a graph definition from a YAML file or similar. This is useful when migrating to Dagster from other workflow systems.

For example, you can have a YAML like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

name: some_example
description: blah blah blah
ops:
  - def: add_one
    alias: A
  - def: add_one
    alias: B
    deps:
      num:
        op: A
  - def: add_two
    alias: C
    deps:
      num:
        op: A
  - def: subtract
    deps:
      left:
        op: B
      right:
        op: C

A DSL is a good way to empower less Pythonic users, such as data analysts. It provides a declarative approach to building data pipelines.

A comprehensive write-up by Elliot Gunn and Tim Castillo on Standardize Pipelines with Domain-Specific Languages | Dagster Blog offers valuable insights.

• Abstracting the Pipelines for Analysts with a YAML DSL | Dagster Blog
• YAML engineer: From Data Engineer to YAML Engineer

Origin:
References:
Created 2022-09-22