[ { "path": ".github/validate-examples", "content": "#!/bin/bash\n\nset -ex\n\n#function datacontract() {\n# docker run --rm -v \"${PWD}:/home/datacontract\" --platform linux/amd64 datacontract/cli:latest \"$@\"\n#}\n\ndatacontract --version\n\nSCHEMA=datacontract.schema.json\n\nawk '/^```yaml$/{flag=1; next} /^```$/{print \"\"; flag=0; exit} flag' README.md > datacontract-from-readme.yaml\ndatacontract lint datacontract-from-readme.yaml --schema $SCHEMA\ndatacontract test --examples datacontract-from-readme.yaml --schema $SCHEMA\n# Compare with example?\n\ndatacontract lint examples/orders-latest/datacontract.yaml --schema $SCHEMA\ndatacontract test --examples examples/orders-latest/datacontract.yaml --schema $SCHEMA\n\ndatacontract lint examples/orders-latest-nested/datacontract.yaml --schema $SCHEMA\ndatacontract test --examples examples/orders-latest-nested/datacontract.yaml --schema $SCHEMA || true # examples are not nested\n\ndatacontract lint examples/covid-cases/datacontract.yaml --schema $SCHEMA\ndatacontract test --examples examples/covid-cases/datacontract.yaml --schema $SCHEMA || true\n\n" }, { "path": ".github/workflows/ci.yaml", "content": "on:\n push:\n pull_request:\n workflow_call:\n\nname: CI\njobs:\n test:\n if: false # skip as the example structure has changed with v1.1.0\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - name: Set up Python\n uses: actions/setup-python@v5\n with:\n python-version: 3.11\n - name: Install dependencies\n run: |\n python -m pip install --upgrade pip\n pip install datacontract-cli[all]\n datacontract --version\n - name: Validate examples\n run: .github/validate-examples\n" }, { "path": ".gitignore", "content": ".idea/\n*.bkp\ndatacontract.schema.openapi-format.*\n.soda/\ndatacontract-from-readme.yaml\n.duckdb/\n" }, { "path": "CHANGELOG.md", "content": "# Changelog\n\nAll notable changes to this project will be documented in this file.\n\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),\nand this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).\n\n\n## [1.2.1] - 2025-09-24\n\n### Added\n\n- Support for data quality metrics that align with ODCS 3.1\n\n### Changed\n- Replaced threshold operators mustBeGreaterThanOrEqualTo with mustBeGreaterOrEqualTo and mustBeLessThanOrEqualTo with mustBeLessOrEqualTo to align with ODCS 3.1, even if it feels wrong...\n\n## [1.2.0] - 2025-07-05\n\n### Added\n- Support for `models.additionalFields` to define if additional fields (columns) are allowed or not in the physical server ([#99](https://github.com/datacontract/datacontract-specification/pull/99))\n- Add `time` data type ([#123](https://github.com/datacontract/datacontract-specification/issues/123))\n- Added `variant` data type ([#113](https://github.com/datacontract/datacontract-specification/issues/113))\n- Added `json` data types ([#112](https://github.com/datacontract/datacontract-specification/issues/112))\n\n### Changed\n- `server.type` changed from enum to simple string to support custom types ([#107](https://github.com/datacontract/datacontract-specification/pull/107))\n\n## [1.1.0] - 2024-10-30\n\n### Added\n- Data quality on model and field level ([#55](https://github.com/datacontract/datacontract-specification/issues/55))\n- Lineage support ([#90](https://github.com/datacontract/datacontract-specification/issues/90))\n- Field and definition `examples` as array of any type, instead of `example` as a single value ([#29](https://github.com/datacontract/datacontract-specification/issues/29)\n- Support for server-specific data types as config map ([#63](https://github.com/datacontract/datacontract-specification/issues/63))\n- AWS Glue Catalog server support\n- sftp server support\n- info.status field\n- oracle server support\n- field.title attribute\n- model.title attribute\n- AWS Kinesis Data Streams server support\n- field.links attribute\n- Trino support\n- Field `type: map` support with properties `keys` and `values`\n- Definitions: `fields`, for type `object`, `record`, and `struct`\n- Field `field.primaryKey` (Replaces `field.primary`)\n- Field `model.primaryKey` to describe a composite primary key\n- Add Redshift server properties `clusterIdentifier`, `endpoint`, `host` and `port`.\n\n### Removed\n\n- `definitions.domain` removed (use a hierarchical structure instead)\n- `definitions.name` removed (use a hierarchical structure instead)\n- `quality` on top-level removed\n- `examples` on top-level removed\n- `schema` removed in favor of encoding any physical schema configuration in the `model` using the `config` map at the field level and supporting import/export ([#21](https://github.com/datacontract/datacontract-specification/issues/21)).\n\n### Deprecated\n\n- `field.primary` (use `field.primaryKey` instead)\n\n\n## [0.9.3] - 2024-03-06\n\n### Added\n\n- Service levels as a top level `servicelevels` element\n- pubsub server support\n- primary key and relationship support via `field.primary` and `field.references` attributes\n- databricks server support improved\n\n## [0.9.2] - 2024-01-04\n\n### Added\n\n- Format and validation attributes to fields in models and definitions\n- Postgres support\n- Databricks support\n\n## [0.9.1] - 2023-11-19\n\n### Added\n\n- A logical data model (#13), mainly to simplify editor support with a defined schema, easier to detect breaking changes, and better Databricks support.\n- Definitions (#14) for reusable semantic definitions within one data contract or across data contracts. \n\n### Removed\n\n- Property `info.dataProduct` as data products should define which data contracts they implement.\n- Property `info.outputPort` as data products should define which data contracts they implement.\n\nThose removals are not considered as breaking changes, as these attributes are now treated as specification extensions.\n\n## [0.9.0] - 2023-09-12\n\nFirst public release.\n" }, { "path": "CNAME", "content": "datacontract-specification.com" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2023 Data Mesh Architecture\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "# Data Contract Specification \n\n\n \"Stars\"\n\"Slack\n\n\n> **Deprecation Notice** \n> With the release of the [Open Data Contract Standard v3.1.0](https://github.com/bitol-io/open-data-contract-standard), we deprecate the Data Contract Specification in line with our commitment to focus on a single industry standard for data contracts. We have actively contributed to the Open Data Contract Standard in the TSC and will continue to support it.

\n> If you are using Data Contract Specification, we recommend [migrating to the Open Data Contract Standard](#migration) within the next few months.
\n> The Data Contract Specification will be supported in Data Contract CLI and Entropy Data until the end of 2026.\n\n![datacontract.png](images/datacontract.png)\n\nData contracts bring data providers and data consumers together.\n\nA _data contract_ is a document that defines the structure, format, semantics, quality, and terms of use for exchanging data between a data provider and their consumers. \nThink of an API, but for data. \nA data contract is implemented by a data product or other data technologies, even legacy data warehouses. \nData contracts can also be used for the input port to specify the expectations of data dependencies and verify given guarantees.\n\nThe _data contract specification_ defines a YAML format to describe attributes of provided data sets. \nIt is data platform neutral and can be used with any data platform, such as AWS S3, Google BigQuery, Azure, Databricks, and Snowflake. \nThe data contract specification is an open initiative to define a common data contract format. \nIt follows [OpenAPI](https://www.openapis.org/) and [AsyncAPI](https://www.asyncapi.com/) conventions.\nIf you haven't adopted a YAML format yet, we recommend to start directly with the [Open Data Contract Standard](https://github.com/bitol-io/open-data-contract-standard). It’s considered the conceptual successor and comes highly recommended.\n\nData contracts come into play when data is exchanged between different teams or organizational units, such as in a [data mesh architecture](https://www.datamesh-architecture.com/). \nFirst, and foremost, data contracts are a communication tool to express a common understanding of how data should be structured and interpreted. \nThey make semantic and quality expectations explicit. \nThey are often created collaboratively in [workshops](./workshop.md) together with data providers and data consumers. \nLater in development and production, they also serve as the basis for code generation, testing, schema validations, quality checks, monitoring, access control, and computational governance policies.\n\nThe specification comes along with the [Data Contract CLI](https://github.com/datacontract/datacontract-cli), an open-source tool to develop, validate, and enforce data contracts.\n\n> _Note: The term \"data contract\" refers to a specification that is usually owned by the data provider and thus does not align with a \"contract\" in a legal sense as a mutual agreement between two parties. \n> The term \"contract\" may be somewhat misleading, but it is how it is used by the industry. \n> The mutual agreement between one data provider and one data consumer is the \"data usage agreement\" that refers to a data contract. \n> Data usage agreements have a defined lifecycle, start/end date, and help the data provider to track who accesses their data and for which purposes._\n\nVersion\n---\n\n1.2.1([Changelog](CHANGELOG.md))\n\nExample\n---\n\nView in [Data Contract Catalog](https://datacontract.com/examples/index.html)\n\n```yaml\ndataContractSpecification: 1.2.1\nid: orders-latest\ninfo:\n title: Orders Latest\n version: 2.0.0\n description: |\n Successful customer orders in the webshop.\n All orders since 2020-01-01.\n Orders with their line items are in their current state (no history included).\n owner: Checkout Team\n status: active\n contact:\n name: John Doe (Data Product Owner)\n url: https://teams.microsoft.com/l/channel/example/checkout\nservers:\n production:\n type: s3\n environment: prod\n location: s3://datacontract-example-orders-latest/v2/{model}/*.json\n format: json\n delimiter: new_line\n description: \"One folder per model. One file per day.\"\n roles:\n - name: analyst_us\n description: Access to the data for US region\n - name: analyst_cn\n description: Access to the data for China region\nterms:\n usage: |\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: |\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n policies:\n - name: privacy-policy\n url: https://example.com/privacy-policy\n - name: license\n description: External data is licensed under agreement 1234.\n url: https://example.com/license/1234\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table\n fields:\n order_id:\n $ref: '#/definitions/order_id'\n required: true\n unique: true\n primaryKey: true\n order_timestamp:\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n type: timestamp\n required: true\n examples:\n - \"2024-09-09T08:30:00Z\"\n tags: [\"business-timestamp\"]\n order_total:\n description: Total amount the smallest monetary unit (e.g., cents).\n type: long\n required: true\n examples:\n - 9999\n quality:\n - type: sql\n description: 95% of all order total values are expected to be between 10 and 499 EUR.\n query: |\n SELECT quantile_cont(order_total, 0.95) AS percentile_95\n FROM orders\n mustBeBetween: [1000, 49900]\n customer_id:\n description: Unique identifier for the customer.\n type: text\n minLength: 10\n maxLength: 20\n customer_email_address:\n description: The email address, as entered by the customer.\n type: text\n format: email\n required: true\n pii: true\n classification: sensitive\n quality:\n - type: text\n description: The email address is not verified and may be invalid.\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: email_address\n processed_timestamp:\n description: The timestamp when the record was processed by the data platform.\n type: timestamp\n required: true\n config:\n jsonType: string\n jsonFormat: date-time\n quality:\n - type: sql\n description: The maximum duration between two orders should be less that 3600 seconds\n query: |\n SELECT MAX(duration) AS max_duration \n FROM (\n SELECT EXTRACT(EPOCH FROM (order_timestamp - LAG(order_timestamp) OVER (ORDER BY order_timestamp))) AS duration \n FROM orders\n )\n mustBeLessThan: 3600\n - type: sql\n description: Row Count\n query: |\n SELECT count(*) as row_count\n FROM orders\n mustBeGreaterThan: 5\n examples:\n - |\n order_id,order_timestamp,order_total,customer_id,customer_email_address,processed_timestamp\n \"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\",\"2030-09-09T08:31:00Z\"\n \"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\",\"2030-09-09T08:31:00Z\"\n \"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\",\"2030-09-09T08:31:00Z\"\n \"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\",\"2030-09-09T08:31:00Z\"\n \"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\",\"2030-09-09T08:31:00Z\"\n \"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\",\"2030-09-09T08:31:00Z\"\n \"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\",\"2030-09-09T08:31:00Z\"\n \"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\",\"2030-09-09T08:31:00Z\"\n line_items:\n description: A single article that is part of an order.\n type: table\n fields:\n line_item_id:\n type: text\n description: Primary key of the lines_item_id table\n required: true\n order_id:\n $ref: '#/definitions/order_id'\n references: orders.order_id\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\n primaryKey: [\"order_id\", \"line_item_id\"]\n examples:\n - |\n line_item_id,order_id,sku\n \"LI-1\",\"1001\",\"5901234123457\"\n \"LI-2\",\"1001\",\"4001234567890\"\n \"LI-3\",\"1002\",\"5901234123457\"\n \"LI-4\",\"1002\",\"2001234567893\"\n \"LI-5\",\"1003\",\"4001234567890\"\n \"LI-6\",\"1003\",\"5001234567892\"\n \"LI-7\",\"1004\",\"5901234123457\"\n \"LI-8\",\"1005\",\"2001234567893\"\n \"LI-9\",\"1005\",\"5001234567892\"\n \"LI-10\",\"1005\",\"6001234567891\"\ndefinitions:\n order_id:\n title: Order ID\n type: text\n format: uuid\n description: An internal ID that identifies an order in the online shop.\n examples:\n - 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n tags:\n - orders\n sku:\n title: Stock Keeping Unit\n type: text\n pattern: ^[A-Za-z0-9]{8,14}$\n examples:\n - \"96385074\"\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article.\n It is typically associated with an article's barcode, such as the EAN/GTIN.\n links:\n wikipedia: https://en.wikipedia.org/wiki/Stock_keeping_unit\n tags:\n - inventory\nservicelevels:\n availability:\n description: The server is available during support hours\n percentage: 99.9%\n retention:\n description: Data is retained for one year\n period: P1Y\n unlimited: false\n latency:\n description: Data is available within 25 hours after the order was placed\n threshold: 25h\n sourceTimestampField: orders.order_timestamp\n processedTimestampField: orders.processed_timestamp\n freshness:\n description: The age of the youngest row in a table.\n threshold: 25h\n timestampField: orders.order_timestamp\n frequency:\n description: Data is delivered once a day\n type: batch # or streaming\n interval: daily # for batch, either or cron\n cron: 0 0 * * * # for batch, either or interval\n support:\n description: The data is available during typical business hours at headquarters\n time: 9am to 5pm in EST on business days\n responseTime: 1h\n backup:\n description: Data is backed up once a week, every Sunday at 0:00 UTC.\n interval: weekly\n cron: 0 0 * * 0\n recoveryTime: 24 hours\n recoveryPoint: 1 week\ntags:\n - checkout\n - orders\n - s3\nlinks:\n datacontractCli: https://cli.datacontract.com\n```\n\nMigration\n---\n\nTo migrate from Data Contract Specification to the Open Data Contract Specification, you can use the [Data Contract CLI](https://github.com/datacontract/datacontract-cli):\n\n```\nuv tool install --python python3.11 --upgrade 'datacontract-cli[all]'\ndatacontract export --format odcs --output odcs.yaml datacontract.yaml\n```\n\nYou can now continue to work with _odcs.yaml_ file.\n\n\n\n\nData Contract CLI\n---\n\nThe [Data Contract CLI](https://cli.datacontract.com) is a command line tool and Python library to lint, test, import and export data contracts (supporting Data Contract Specification and ODCS).\n\nHere is a short example of how to verify that your actual dataset matches the data contract: \n\n```bash\npip3 install \"datacontract-cli[all]\"\ndatacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nor, if you prefer Docker:\n```bash\ndocker run datacontract/cli test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nThe Data Contract contains all required information to verify data: \n\n- The _servers_ block has the connection details to the actual data set.\n- The _models_ define the syntax, formats, and constraints. \n- The _quality_ defined further quality checks.\n\nThe Data Contract CLI chooses the appropriate engine, formulates test cases, connects to the server, and executes the tests, based on the server type.\n\nMore information and configuration options on [cli.datacontract.com](https://cli.datacontract.com).\n\n\n\nSpecification\n---\n\n![The eight major categories in the data contract specification](images/categories.png)\n\n- [Data Contract Object](#data-contract-object)\n- [Info Object](#info-object)\n- [Contact Object](#contact-object)\n- [Server Object](#server-object)\n- [Terms Object](#terms-object)\n- [Model Object](#model-object)\n- [Field Object](#field-object)\n- [Definition Object](#definition-object)\n- [Service Level Object](#service-levels-object)\n- [Quality Object](#quality-object)\n- [Lineage Object](#lineage-object)\n- [Data Types](#data-types)\n- [Specification Extensions](#specification-extensions)\n\n\n[JSON Schema](https://github.com/datacontract/datacontract-specification/blob/main/datacontract.schema.json) of the Data Contract Specification.\n\n### Data Contract Object\n\nThis is the root document.\n\nIt is _RECOMMENDED_ that the root document be named: `datacontract.yaml`.\n\n| Field | Type | Description |\n|---------------------------|--------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|\n| dataContractSpecification | `string` | REQUIRED. Specifies the Data Contract Specification being used. |\n| id | `string` | REQUIRED. An organization-wide unique technical identifier, such as a UUID, URN, slug, string, or number |\n| info | [Info Object](#info-object) | REQUIRED. Specifies the metadata of the data contract. May be used by tooling. |\n| servers | Map[`string`, [Server Object](#server-object)] | Specifies the servers of the data contract. |\n| terms | [Terms Object](#terms-object) | Specifies the terms and conditions of the data contract. |\n| models | Map[`string`, [Model Object](#model-object)] | Specifies the logical data model. |\n| definitions | Map[`string`, [Definition Object](#definition-object)] | Specifies definitions. |\n| servicelevels | [Service Levels Object](#service-levels-object) | Specifies the service level of the provided data |\n| links | Map[`string`, `string`] | Additional external documentation links. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n\n### Info Object\n\nMetadata and life cycle information about the data contract.\n\n\n| Field | Type | Description |\n|-------------|-----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| title | `string` | REQUIRED. The title of the data contract. |\n| version | `string` | REQUIRED. The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version). |\n| status | `string` | The status of the data contract. Can be `proposed`, `in development`, `active`, `deprecated`, `retired`. |\n| description | `string` | A description of the data contract. |\n| owner | `string` | The owner or team responsible for managing the data contract and providing the data. |\n| contact | [Contact Object](#contact-object) | Contact information for the data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Contact Object\n\nContact information for the data contract.\n\n| Field | Type | Description |\n|-------|----------|-------------------------------------------------------------------------------------------------------|\n| name | `string` | The identifying name of the contact person/organization. |\n| url | `string` | The URL pointing to the contact information. This _MUST_ be in the form of a URL. |\n| email | `string` | The email address of the contact person/organization. This _MUST_ be in the form of an email address. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n### Server Object\n\nThe fields are dependent on the defined type.\n\n| Field | Type | Description |\n|-------------|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `clickhouse`, `s3`, `glue`, `redshift`, `azure`, `sqlserver`, `snowflake`, `databricks`, `postgres`, `oracle`, `kafka`, `pubsub`, `sftp`, `kinesis`, `trino`, `local` |\n| description | `string` | An optional string describing the server. |\n| environment | `string` | An optional string describing the environment, e.g., prod, sit, stg. |\n| roles | Array of [Server Role Object](#server-role-object) | An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### BigQuery Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `bigquery` |\n| project | `string` | The GCP project name. |\n| dataset | `string` | |\n\n#### S3 Server Object\n\n| Field | Type | Description |\n|-------------|----------|-------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `s3` |\n| location | `string` | S3 URL, starting with `s3://` |\n| endpointUrl | `string` | The server endpoint for S3-compatible servers, such as MioIO or Google Cloud Storage, e.g., `https://minio.example.com` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\nExample (AWS S3):\n\n```yaml\nservers:\n production:\n type: s3\n location: s3://acme-orders-prod/orders/\n format: json\n delimiter: new_line\n```\n\nExample (MinIO):\n\n```yaml\nservers:\n minio:\n type: s3\n endpointUrl: http://localhost:9000\n location: s3://my-bucket/path/\n format: delta\n```\n\nExample (Google Cloud Storage):\n\n```yaml\nservers:\n gcs:\n type: s3\n endpointUrl: https://storage.googleapis.com\n location: s3://my-bucket/path/*/*/*/*/*.parquet\n format: parquet\n```\n\n\n#### Redshift Server Object\n\n| Field | Type | Description |\n|-------------------|----------|---------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `redshift` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n| clusterIdentifier | `string` | Identifier of the cluster.
Example: `analytics-cluster` |\n| host | `string` | Host of the cluster.
Example: `analytics-cluster.example.eu-west-1.redshift.amazonaws.com` |\n| port | `number` | Port of the cluster.
Example: `5439` |\n| endpoint | `string` | Endpoint of the cluster
Example: `analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics` |\n\nExample, specifying an endpoint:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n endpoint: analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\n```\n\nExample, specifying the cluster identifier:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n clusterIdentifier: analytics-cluster\n```\n\nExample, specifying the cluster host:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n host: analytics-cluster.example.eu-west-1.redshift.amazonaws.com\n port: 5439\n```\n\n#### Azure Server Object\n\n| Field | Type | Description |\n|----------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `azure` |\n| storageAccount | `string` | The storage account name that contains the files |\n| location | `string` | Path to Azure Blob Storage or Azure Data Lake Storage (ADLS) in the storage account, supports globs. Starting with `az://` or `abfss`
Recommended pattern is `abfss:///`, Examples: `az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet` or `abfss://my_container_name/path/*.parquet` |\n| format | `string` | Format of files, such as `parquet`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\n\n#### SQL-Server Server Object\n\n| Field | Type | Description |\n|----------|-----------|--------------------------------------------------------------------------|\n| type | `string` | `sqlserver` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server, default: `1433` |\n| database | `string` | The name of the database, e.g., `database`. |\n| schema | `string` | The name of the schema in the database, e.g., `dbo`. |\n| driver | `string` | The name of the supported driver, e.g., `ODBC Driver 18 for SQL Server`. |\n\n\n#### Snowflake Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `snowflake` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Databricks Server Object\n\n| Field | Type | Description |\n|---------|----------|---------------------------------------------------------------------|\n| type | `string` | `databricks` |\n| host | `string` | The Databricks host, e.g., `dbc-abcdefgh-1234.cloud.databricks.com` |\n| catalog | `string` | The name of the Hive or Unity catalog |\n| schema | `string` | The schema name in the catalog |\n\n#### Postgres Server Object\n\n| Field | Type | Description |\n|----------|-----------|---------------------------------------------------------|\n| type | `string` | `postgres` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server |\n| database | `string` | The name of the database, e.g., `postgres`. |\n| schema | `string` | The name of the schema in the database, e.g., `public`. |\n\n#### Oracle Server Object\n\n| Field | Type | Description |\n|-------------|-----------|---------------------------------|\n| type | `string` | `oracle` |\n| host | `string` | The host to the oracle server |\n| port | `integer` | The port to the oracle server |\n| serviceName | `string` | The name of the service |\n\n#### Kafka Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kafka` |\n| host | `string` | The bootstrap server of the kafka cluster. |\n| topic | `string` | The topic name. |\n| format | `string` | The format of the message. Examples: json, avro, protobuf. Default: json. |\n\n#### Pub/Sub Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `pubsub` |\n| project | `string` | The GCP project name. |\n| topic | `string` | The topic name. |\n\n#### sftp Server Object\n\n| Field | Type | Description |\n|-----------|----------|------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `sftp` |\n| location | `string` | SFTP URL, starting with `sftp://` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\n#### AWS Kinesis Data Streams Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kinesis` |\n| stream | `string` | The name of the Kinesis data stream. |\n| region | `string` | AWS region, e.g., `eu-west-1`. |\n| format | `string` | The format of the records. Examples: json, avro, protobuf. |\n\n#### Trino Server Object\n\n| Field | Type | Description |\n|----------|-----------|-----------------------------------------------------------|\n| type | `string` | `trino` |\n| host | `string` | The Trino host |\n| port | `integer` | The Trino port | \n| catalog | `string` | The name of the catalog, e.g., `my_catalog`. |\n| schema | `string` | The name of the schema in the catalog, e.g., `my_schema`. |\n\n#### Local Server Object\n\n| Field | Type | Description |\n|--------|----------|-------------------------------------------------------------------------------------|\n| type | `string` | `local` |\n| path | `string` | The relative or absolute path to the data file(s), such as `./folder/data.parquet`. |\n| format | `string` | The format of the file(s), such as `parquet`, `delta`, `csv`, or `json`. |\n\n#### Server Role Object\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------|\n| name | `string` | Name of the role |\n| description | `string` | A description of the role and what access the role provides. |\n\n### Terms Object\n\nThe terms and conditions of the data contract.\n\n| Field | Type | Description |\n|--------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| usage | `string` | The usage describes the way the data is expected to be used. Can contain business and technical information. |\n| limitations | `string` | The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for. |\n| policies | Array of [Policy Object](#policy-object) | A list of policies, licenses, standards, that are applicable for this data contract and that must be acknowledged by data consumers. |\n| billing | `string` | The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use. |\n| noticePeriod | `string` | The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., `P3M` for a period of three months. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Policy Object\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------|\n| name | `string` | Name of the policy. |\n| description | `string` | A description of the policy. |\n| url | `string` | An URL that refers to the policy. |\n\n\n### Model Object\n\nThe Model Object describes the structure and semantics of a data model, such as tables, views, or structured files.\n\nThe name of the data model (table name) is defined by the key that refers to this Model Object.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the model. Examples: `table`, `view`, `object`. Default: `table`. |\n| description | `string` | An optional string describing the data model. |\n| title | `string` | An optional string for the title of the data model. Especially useful if the name of the model is cryptic or contains abbreviations. |\n| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |\n| primaryKey | Array of `string` | If the primary key is a compound key, list the field names that constitute the primary key. Alternative to field-level `primaryKey`. |\n| quality | Array of [Quality Object](#quality-object) | Specifies the quality attributes on model level. |\n| examples | Array of `Any` | Specifies example data sets for the model. |\n| additionalFields | `Boolean` | Specify, if the model can have additional fields that are not defined in the contract. Default: `false`. |\n| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |\n\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Field Object\n\nThe Field Objects describes one field (column, property, nested field) of a data model.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the semantic of the data in this field. |\n| type | [Data Type](#data-types) | The logical data type of the field. |\n| title | `string` | An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations. |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| required | `boolean` | An indication, if this field must contain a value and may not be null. Default: `false` |\n| primaryKey | `boolean` | If this field is a primary key. Default: `false` |\n| references | `string` | The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship. |\n| unique | `boolean` | An indication, if the value must be unique within the model. Default: `false` |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| precision | `number` | The maximum number of digits in a number. Only applies to numeric values. Defaults to 38. |\n| scale | `number` | The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0. |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| ~~example~~ | `string` | DEPRECATED, use examples. An example value. |\n| examples | Array of Any | A list of example values. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). | \n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. Examples may be: `sensitive`, `restricted`, `internal`, `public`. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| links | Map[`string`,`string`] | Additional external documentation links. |\n| $ref | `string` | A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition. |\n| fields | Map[`string`, [Field Object](#field-object)] | The nested fields (e.g. columns) of the object, record, or struct. Use only when type is `object`, `record`, or `struct`. |\n| items | [Field Object](#field-object) | The type of the elements in the array. Use only when type is `array`. |\n| keys | [Field Object](#field-object) | Describes the key structure of a map. Defaults to `type: string` if a map is defined as type. Not all server types support different key types. Use only when type is `map`. |\n| values | [Field Object](#field-object) | Describes the value structure of a map. Use only when type is `map`. |\n| quality | Array of [Quality Object](#quality-object) | Specifies the quality attributes on field level. |\n| lineage | [Lineage Object](#lineage-object) | Provides information where the data comes from. |\n| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Definition Object\n\nThe Definition Object includes a clear and concise explanations of syntax, semantic, and classification of a business object in a given domain.\nIt serves as a reference for a common understanding of terminology, ensure consistent usage and to identify join-able fields.\nModels fields can refer to definitions using the `$ref` field to link to existing definitions and avoid duplicate documentations.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | [Data Type](#data-types) | REQUIRED. The logical data type |\n| title | `string` | The business name of this definition. |\n| description | `string` | Clear and concise explanations related to the domain |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| precision | `number` | The maximum number of digits in a number. Only applies to numeric values. Defaults to 38. |\n| scale | `number` | The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0. |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| examples | Array of Any | A list of example values. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). |\n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| links | Map[`string`, `string`] | Additional external documentation links. |\n| fields | Map[`string`, [Field Object](#field-object)] | The nested fields (e.g. columns) of the object, record, or struct. Use only when type is `object`, `record`, or `struct`. |\n| items | [Field Object](#field-object) | The type of the elements in the array. Use only when type is `array`. |\n| keys | [Field Object](#field-object) | Describes the key structure of a map. Defaults to `type: string` if a map is defined as type. Not all server types support different key types. Use only when type is `map`. |\n| values | [Field Object](#field-object) | Describes the value structure of a map. Use only when type is `map`. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n### Service Levels Object\n\nA service level is defined as an agreed-upon, measurable level of performance for provided the data.\nData Contract Specification defines well-known service levels. \nThis list can be extended with custom service levels.\n\nOne can either describe each service level informally using the `description` field, or make use of the predefined fields for automation support, e.g., via the [Data Contract CLI](https://cli.datacontract.com).\n\n| Field | Type | Description |\n|--------------|-----------------------------------------------|-------------------------------------------------------------------------|\n| availability | [Availability Object](#availability-object) | The promised uptime of the system that provides the data |\n| retention | [Retention Object](#retention-object) | The period how long data will be available. |\n| latency | [Latency Object](#latency-object) | The maximum amount of time from the source to its destination. |\n| freshness | [Freshness Object](#freshness-object) | The maximum age of the youngest entry. |\n| frequency | [Frequency Object](#frequency-object) | The update frequency. |\n| support | [Support Object](#support-object) | The times when support is provided. |\n| backup | [Backup Object](#backup-object) | The details about data backup procedures. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Availability Object\n\nAvailability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the availability service level. |\n| percentage | `string` | An optional string describing the guaranteed uptime in percent (e.g., `99.9%`) |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Retention Object\n\nRetention covers the period how long data will be available.\n\n| Field | Type | Description |\n|----------------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the retention service level. |\n| period | `string` | An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`). |\n| unlimited | `boolean` | An optional indicator that data is kept forever. |\n| timestampField | `string` | An optional reference to the field that contains the timestamp that the period refers to. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Latency Object\n\nLatency refers to the maximum amount of time from the source to its destination.\n\nExamples are the maximum duration it takes after an order has been recorded in the ecommerce shop until it is available in the orders table in the data analytics platform. This includes the waiting times until the next batch run is started and the processing time of the pipeline.\n\n| Field | Type | Description |\n|-------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the latency service level. |\n| threshold | `string` | An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`). |\n| sourceTimestampField | `string` | An optional reference to the field that contains the timestamp when the data was provided at the source. |\n| processedTimestampField | `string` | An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Freshness Object\n\nFreshness refers to the maximum age of the youngest entry.\n\n| Field | Type | Description |\n|-------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the freshness service level. |\n| threshold | `string` | An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`). |\n| timestampField | `string` | An optional reference to the field that contains the timestamp that the threshold refers to. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Frequency Object\n\nFrequency describes how often data is updated.\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the frequency service level. |\n| type | `string` | An optional type of data processing. Typical values are `batch`, `micro-batching`, `streaming`, `manual`. |\n| interval | `string` | Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`. |\n| cron | `string` | Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n#### Support Object\n\nSupport describes the times when support will be available for contact.\n\n| Field | Type | Description |\n|--------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the support service level. |\n| time | `string` | An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`. |\n| responseTime | `string` | An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with. |\n \n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n#### Backup Object\n\nBackup specifies details about data backup procedures.\n\n| Field | Type | Description |\n|---------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the backup service level. |\n| interval | `string` | An optional interval that defines how often data will be backed up, e.g., `daily`. |\n| cron | `string` | An optional cron expression when data will be backed up, e.g., `0 0 * * *`. |\n| recoveryTime | `string` | An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours). |\n| recoveryPoint | `string` | An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours). |\n\n\n\n### Quality Object\n\nThe quality object defines quality attributes.\n\nQuality attributes are checks that can be applied to the data to ensure its quality. \nData can be verified by executing these checks through a data quality engine.\n\nQuality attributes can be:\n- A text in natural language that describes the quality of the data.\n- A predefined metric from the library of commonly used metrics\n- An individual SQL query that returns a single value that can be compared.\n- Engine-specific types: Pre-defined quality checks, as defined by data quality libraries. Currently, the engines `soda` and `great-expectations` are supported.\n\nA quality object can be specified on the field level and on the model level. \nThe top-level quality object is deprecated.\n\n#### Description Text\n\nA description in natural language that defines the expected quality of the data. \nThis is useful to express requirements or expectations when discussing the data contract with stakeholders.\nLater in the development process, these might be translated into an executable check (such as `sql`).\nIt can also be used as a prompt to check the data with an AI engine.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------|\n| type | `string` | `text` |\n| description | `string` | A plain text describing the quality attribute in natural language. |\n\nExample:\n\n```yaml\nmodels:\n my_table:\n fields:\n account_iban:\n quality:\n - type: text\n description: Must be a valid IBAN. Must not be empty.\n```\n\n#### SQL\n\nAn individual SQL query that returns a single number that can be compared with a threshold. The SQL query must be in the SQL dialect of the provided server.\n\n> __Note:__ Establish a secure development process and use read-only connections, as the misuse of SQL queries can lead to SQL injection attacks.\n\n\n| Field | Type | Description |\n|----------------------------|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `sql` |\n| description | `string` | A plain text describing the quality of the data. |\n| query | `string` | A SQL query that returns a single number to compare with the threshold. |\n| dialect | `string` | The SQL dialect that is used for the query. Should be compatible to the server type. Examples: `postgres`, `spark`, `bigquery`, `snowflake`, `duckdb`, ... |\n| mustBe | `integer` | The threshold to check the return value of the query |\n| mustNotBe | `integer` | The threshold to check the return value of the query |\n| mustBeGreaterThan | `integer` | The threshold to check the return value of the query |\n| mustBeGreaterThanOrEqualTo | `integer` | The threshold to check the return value of the query |\n| mustBeLessThan | `integer` | The threshold to check the return value of the query |\n| mustBeLessThanOrEqualTo | `integer` | The threshold to check the return value of the query |\n| mustBeBetween | array of two integers | The threshold to check the return value of the query. Boundaries are inclusive. |\n| mustNotBeBetween | array of two integers | The threshold to check the return value of the query. Boundaries are inclusive. |\n\nIn the query the following placeholders can be used:\n\n| Placeholder | Description |\n|-------------|----------------------------------------------------------------------------------------|\n| `{model}` | The name of the model that is checked. |\n| `{table}` | Alias for `{model}`. |\n| `{field}` | The name of the field that is checked (only if the quality is defined on field-level). |\n| `{column}` | Alias for `{field}`. |\n\nExample:\n\n```yaml\nmodels:\n orders:\n quality:\n - type: sql\n description: The maximum duration between two orders must be less that 3600 seconds\n query: |\n SELECT MAX(duration) AS max_duration \n FROM (\n SELECT EXTRACT(EPOCH FROM (order_timestamp - LAG(order_timestamp) OVER (ORDER BY order_timestamp))) AS duration \n FROM {model}\n )\n mustBeLessThan: 3600\n```\n\nSQL queries allow powerful checks for custom business logic. \nA SQL query should run not longer than 10 minutes.\n\n#### Library / Metrics\n\nA set of predefined metrics commonly used in data quality checks, designed to be compatible with all major data quality engines. This simplifies the work for data engineers by eliminating the need to manually write SQL queries.\nThese metrics are aligned with ODCS 3.1.\n\n| Field | Type | Description |\n|------------------------|-----------------------|----------------------------------------------------------------------------------|\n| type | `string` | `library` (can be omitted, if `metric` is defined) |\n| metric | `string` | `nullValues`, `missingValues`, `invalidValues`, `duplicateValues`, or `rowCount` |\n| arguments | `object` | Some metrics require additional arguments |\n| description | `string` | A plain text describing the quality of the data. |\n| mustBe | `integer` | The threshold to check the return value of the query |\n| mustNotBe | `integer` | The threshold to check the return value of the query |\n| mustBeGreaterThan | `integer` | The threshold to check the return value of the query |\n| mustBeGreaterOrEqualTo | `integer` | The threshold to check the return value of the query |\n| mustBeLessThan | `integer` | The threshold to check the return value of the query |\n| mustBeLessOrEqualTo | `integer` | The threshold to check the return value of the query |\n| mustBeBetween | array of two integers | The threshold to check the return value of the query. Boundaries are inclusive. |\n| mustNotBeBetween | array of two integers | The threshold to check the return value of the query. Boundaries are inclusive. |\n| unit | `string` | `rows` (default) or `percent` |\n\n\nMetrics:\n\n| Metric | Level | Description | Arguments | Arguments Example |\n|--------|--------|----------------------------------------------------------------|------------------------------------------------------------------|----------------------------------------------------------------------|\n| `nullValues` | Property | Counts null values in a column/field | None | |\n| `missingValues` | Property | Counts values considered as missing (empty strings, N/A, etc.) | `missingValues`: Array of values considered missing | `missingValues: [null, '', 'N/A']` |\n| `invalidValues` | Property | Counts values that don't match valid criteria | `validValues`: Array of valid values
`pattern`: Regex pattern | `validValues: ['pounds', 'kg']`
`pattern: '^[A-Z]{2}[0-9]{2}...'` |\n| `duplicateValues` | Property | Counts duplicate values in a column | None | |\n| `duplicateValues` | Schema | Counts duplicate values across multiple columns | `properties`: Array of property names | `properties: ['tenant_id', 'order_id']` |\n| `rowCount` | Schema | Counts total number of rows in a table/object store | None | |\n\n\nExample:\n\n```yaml\nproperties:\n - name: email_address\n quality:\n - metric: missingValues\n arguments:\n missingValues: [null, '', 'N/A', 'n/a']\n mustBeLessThan: 5\n unit: percent # rows (default) or percent\n```\n\n\n#### Custom\n\nYou can define custom quality attributes that are specific to a data quality engine.\n\n#### Custom (Engine: Soda)\n\nSoda has a number of predefined quality [checks](https://docs.soda.io/soda/data-contracts-checks.html) that can be referenced as quality attributes.\n\nSoda checks can be applied on model and field level.\n\n> Note: Soda Data contract check reference is experimental and may change in the future. Currently only supported by Postgres, Snowflake, and Spark (Databricks)\n\n| Field | Type | Description |\n|---------------|----------|-----------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `custom` |\n| description | `string` | Optional. A plain text describing the quality attribute in natural language. |\n| engine | `string` | `soda` |\n| implementation | `object` | A check type as defined in the [Data contract check reference](https://docs.soda.io/soda/data-contracts-checks.html) |\n\n\nSee the [Data contract check reference](https://docs.soda.io/soda/data-contracts-checks.html) for all possible types and configuration values.\n\n\nExample:\n\n```yaml\nmodels:\n my_table:\n fields:\n order_id:\n type: string\n quality:\n - type: custom\n description: This is a check on field level\n engine: soda\n implementation:\n type: no_duplicate_values\n carrier:\n type: string\n shipment_numer:\n type: string\n quality:\n - type: custom\n description: This is a check on model level\n engine: soda\n implementation:\n type: duplicate_percent\n columns:\n - carrier\n - shipment_numer\n must_be_less_than: 1.0\n - type: custom\n description: This is a check on model level\n engine: soda\n implementation:\n type: row_count\n must_be_greater_than: 500000\n```\n\n#### Custom (Engine: Great Expectations)\n\nQuality attributes defined as Great Expectations [Expectation](https://greatexpectations.io/expectations/).\n\nExpectations are applied on model level.\n\n| Field | Type | Description |\n|---------------|----------|-----------------------------------------------------------------------------------------------------|\n| description | `string` | Optional. A plain text describing the quality attribute in natural language. |\n| engine | `string` | `great-expectations` |\n| implementation | `object` | An expectation type as listed in [Expectation](https://greatexpectations.io/expectations/) as YAML. |\n\nExample:\n\n```yaml\nmodels:\n my_table:\n quality:\n - type: custom\n engine: great-expectations\n implementation:\n expectation_type: expect_table_row_count_to_be_between\n kwargs:\n min_value: 10000\n max_value: 50000\n meta:\n notes: \"This expectation is crucial to avoid processing datasets that are too small or too large.\"\n - type: custom\n engine: great-expectations\n description: \"Check that passenger_count values are between 1 and 6.\"\n implementation:\n expectation_type: expect_column_values_to_be_between\n kwargs:\n column: passenger_count\n max_value: 6\n min_value: 1\n mostly: 1.0\n strict_max: false\n strict_min: false\n meta:\n tags:\n - business-critical\n - range_check\n```\n\n\n### Lineage Object\n\nField level lineage provides optional fine-grained information where the data comes from and how it was transformed.\n\nThe lineage object is based on the OpenLinage [Column Level Lineage Dataset Facet](https://openlineage.io/docs/spec/facets/dataset-facets/column_lineage_facet) to describe the input fields.\n\n\n\n| Field | Type | Description |\n|-------------|---------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| inputFields | Array of [InputField Object](#inputfield-object) | The input fields refer to specific fields, columns, or data points from source systems or other data contracts that feed into a particular transformation, calculation, or final result. |\n\n\n#### InputField Object\n\n| Field | Type | Description |\n|-----------------|----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| namespace | `string` | The input dataset namespace, such as the name of the source system or the domain of another data contract. Examples: `com.example.crm`, `checkout`, snowflake://{account name}. [More on namespace](https://openlineage.io/blog/whats-in-a-namespace/#namespaces-in-the-spec) |\n| name | `string` | The input dataset name, such as a reference to a data contract, a fully qualified table name, a Kafka topic. |\n| field | `string` | The input field name, such as the field in an upstream data contract, a table column or a JSON Path. |\n| transformations | Array of [Transformation Object](#transformation-object) | Optional. This describes how the input field data was used to generate the final result. |\n\n#### Transformation Object\n\n| Field | Type | Description |\n|-------------|-----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | Indicates how direct is the relationship e.g. in query. Allows values are: `DIRECT` and `INDIRECT`. |\n| subtype | `string` | Optional. Contains more specific information about the transformation.
Allowed values for type `DIRECT`: `IDENTITY`, `TRANSFORMATION`, `AGGREGATION`.
Allowed values for type `INDIRECT`: `JOIN`, `GROUP_BY`, `FILTER`, `SORT`, `WINDOW`, `CONDITIONAL`. |\n| description | `string` | Optional. A string representation of the transformation applied. |\n| masking | `boolean` | Optional. Boolean value indicating if the input value was obfuscated during the transformation. |\n\n\nExample:\n\n```yaml\nmodels:\n orders:\n fields:\n order_id:\n type: string\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: order_id\n transformations:\n - type: DIRECT\n subtype: IDENTITY\n description: The order ID from the checkout order\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: order_timestamp\n transformations:\n - type: INDIRECT\n subtype: SORT\n customer_email_address_hash:\n type: string\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: email_address\n transformations:\n - type: DIRECT\n subtype: Transformation\n description: The email address from the checkout order, hashed with SHA-256\n masking: true\n\n```\n\n\n\n### Config Object\n\nThe config field can be used to set additional metadata that may be used by tools, e.g. to define a namespace for code generation, specify physical data types, toggle tests, etc.\n\nA config field can be added with any name. The value can be null, a primitive, an array or an object.\n\nFor developer experience, a list of well-known field names is maintained here, as these fields are used in the Data Contract CLI:\n\n\n| Field | Type | Description |\n|-----------------|----------|----------------------------------------------------------------------------------------------------------------|\n| avroNamespace | `string` | (Only on model level) The namespace to use when importing and exporting the data model from / to Apache Avro. |\n| avroType | `string` | (Only on field level) Specify the field type to use when exporting the data model to Apache Avro. |\n| avroLogicalType | `string` | (Only on field level) Specify the logical field type to use when exporting the data model to Apache Avro. |\n| bigqueryType | `string` | (Only on field level) Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)` |\n| snowflakeType | `string` | (Only on field level) Specify the physical column type that is used in a Snowflake table, e.g, `TIMESTAMP_LTZ` |\n| redshiftType | `string` | (Only on field level) Specify the physical column type that is used in a Redshift table, e.g, `SMALLINT` |\n| sqlserverType | `string` | (Only on field level) Specify the physical column type that is used in a SQL Server table, e.g, `DATETIME2` |\n| databricksType | `string` | (Only on field level) Specify the physical column type that is used in a Databricks table |\n| glueType | `string` | (Only on field level) Specify the physical column type that is used in a AWS Glue Data Catalog table |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\nExample:\n\n```\nmodels:\n orders:\n config:\n avroNamespace: \"my.namespace\"\n fields:\n my_field_1:\n description: Example for AVRO with Timestamp (millisecond precision)\n type: timestamp\n config:\n avroType: long\n avroLogicalType: timestamp-millis\n snowflakeType: timestamp_tz\n```\n\n\n### Data Types\n\nThe following data types are supported for model fields and definitions:\n\n- Unicode character sequence: `string`, `text`, `varchar`\n- Any numeric type, either integers or floating point numbers: `number`, `decimal`, `numeric`\n- 32-bit signed integer: `int`, `integer`\n- 64-bit signed integer: `long`, `bigint`\n- Single precision (32-bit) IEEE 754 floating-point number: `float`\n- Double precision (64-bit) IEEE 754 floating-point number: `double`\n- Binary value: `boolean`\n- Timestamp with timezone: `timestamp`, `timestamp_tz`\n- Timestamp with no timezone: `timestamp_ntz`\n- Date with no time information: `date`\n- Time with no date information: `time`\n- Array: `array`\n- Map: `map` (may not be supported by some server types)\n- Sequence of 8-bit unsigned bytes: `bytes`\n- Complex type: `object`, `record`, `struct`\n- Semi-structured data: `variant` (may not be supported by some server types)\n- JSON data: `json` (may not be supported by some server types)\n- No value: `null`\n\n### Specification Extensions\n\nWhile the Data Contract Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points.\n\nA custom field can be added with any name. The value can be null, a primitive, an array or an object. \n\n\nTooling\n---\n- [Data Contract CLI](https://github.com/datacontract/datacontract-cli) is an open-source CLI tool to help you create, develop, and maintain your data contracts.\n- [Data Contract Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data contracts. It includes a data contract catalog, a Web-Editor, and a request and approval workflow to automate access to data products for a full enterprise data marketplace. \n- [Data Contract GPT](https://gpt.datacontract.com) is a custom GPT that can help you write data contracts.\n- [Data Contract Editor](https://editor.datacontract.com) is an open-source editor for Data Contracts, including a live html preview.\n\nCode Completion\n---\nThe [JSON Schema](https://datacontract.com/datacontract.schema.json) of the current data contract specification is registered in [Schema Store](https://www.schemastore.org/), which brings code completion and syntax checks for all major IDEs.\nIntelliJ comes with a built-in YAML plugin which will show you autocompletions. \nFor VS Code we recommend to install the [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) plugin. \nNo additional configuration is required. \n\nAutocompletion is then enabled for files following these patterns:\n\n```\ndatacontract.yaml\ndatacontract.yml\n*-datacontract.yaml\n*-datacontract.yml\n*.datacontract.yaml\n*.datacontract.yml\ndatacontract-*.yaml\ndatacontract-*.yml\n**/datacontract/*.yml\n**/datacontract/*.yaml\n**/datacontracts/*.yml\n**/datacontracts/*.yaml\n```\n\n\nAuthors\n---\nThe Data Contract Specification was originally created by [Jochen Christ](https://www.linkedin.com/in/jochenchrist/) and [Dr. Simon Harrer](https://www.linkedin.com/in/simonharrer/), and is currently maintained by them.\n\n\nContributing\n---\nContributions are welcome! Please open an issue or a pull request.\n\nLicense\n---\n[MIT License](LICENSE)\n\n\n\n" }, { "path": "_config.yml", "content": "plugins:\n - jekyll-sitemap\nname: Data Contract Specification\ntitle: null\ndescription: Data contracts bring data providers and data consumers together.\n" }, { "path": "_layouts/default.html", "content": "\n\n \n \n \n\n \n \n \n \n\n \n\n {% seo %}\n \n \n \n \n
\n {% if site.title and site.title != page.title %}\n

{{ site.title }}

\n {% endif %}\n\n {{ content }}\n\n {% if site.github.private != true and site.github.license %}\n
\n This site is open source. {% github_edit_link \"Improve this page\" %}.\n
\n {% endif %}\n
\n \n \n \n {% if site.google_analytics %}\n \n {% endif %}\n \n \n \n \n\n" }, { "path": "datacontract.init.yaml", "content": "dataContractSpecification: 1.2.1\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# production:\n# type: s3\n# location: s3://\n# format: parquet\n# delimiter: new_line\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### models\n\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type:\n# description:\n\n\n### definitions\n\n# definitions:\n# my_field:\n# domain:\n# name:\n# title:\n# type:\n# description:\n# example:\n# pii:\n# classification:\n\n\n### servicelevels\n\n#servicelevels:\n# availability:\n# description: The server is available during support hours\n# percentage: 99.9%\n# retention:\n# description: Data is retained for one year because!\n# period: P1Y\n# unlimited: false\n# latency:\n# description: Data is available within 25 hours after the order was placed\n# threshold: 25h\n# sourceTimestampField: orders.order_timestamp\n# processedTimestampField: orders.processed_timestamp\n# freshness:\n# description: The age of the youngest row in a table.\n# threshold: 25h\n# timestampField: orders.order_timestamp\n# frequency:\n# description: Data is delivered once a day\n# type: batch # or streaming\n# interval: daily # for batch, either or cron\n# cron: 0 0 * * * # for batch, either or interval\n# support:\n# description: The data is available during typical business hours at headquarters\n# time: 9am to 5pm in EST on business days\n# responseTime: 1h\n# backup:\n# description: Data is backed up once a week, every Sunday at 0:00 UTC.\n# interval: weekly\n# cron: 0 0 * * 0\n# recoveryTime: 24 hours\n# recoveryPoint: 1 week\n" }, { "path": "datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"title\": \"DataContractSpecification\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"title\": \"DataContractSpecificationVersion\",\n \"enum\": [\n \"1.2.1\",\n \"1.2.0\",\n \"1.1.0\",\n \"0.9.3\",\n \"0.9.2\",\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"status\": {\n \"type\": \"string\",\n \"description\": \"The status of the data contract. Can be proposed, in development, active, retired.\",\n \"examples\": [\n \"proposed\",\n \"in development\",\n \"active\",\n \"deprecated\",\n \"retired\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\",\n \"additionalProperties\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"description\": \"Information about the servers.\",\n \"additionalProperties\": {\n \"$ref\": \"#/$defs/BaseServer\",\n \"allOf\": [\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"bigquery\"\n }\n }\n },\n \"then\": {\n \"$ref\": \"#/$defs/BigQueryServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"s3\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/S3Server\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sftp\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SftpServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"redshift\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/RedshiftServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"azure\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/AzureServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sqlserver\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SqlserverServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"snowflake\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SnowflakeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"databricks\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DatabricksServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"dataframe\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DataframeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"glue\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/GlueServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"oracle\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/OracleServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kafka\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KafkaServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"pubsub\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PubSubServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kinesis\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KinesisDataStreamsServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"trino\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/TrinoServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"clickhouse\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/ClickhouseServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"local\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/LocalServer\"\n }\n }\n ]\n }\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"policies\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the policy.\",\n \"examples\": [\n \"privacy\",\n \"security\",\n \"retention\",\n \"compliance\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the policy.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"A URL to the policy document.\"\n }\n },\n \"additionalProperties\": true\n },\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n },\n \"additionalProperties\": true\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Model\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, view, object. Default: table.\",\n \"type\": \"string\",\n \"title\": \"ModelType\",\n \"default\": \"table\",\n \"enum\": [\n \"table\",\n \"view\",\n \"object\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the model. Especially useful if the model name is cryptic or contains abbreviations.\",\n \"examples\": [\n \"Purchase Orders\",\n \"Air Shipments\"\n ]\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Field\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"required\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if this field must contain a value and may not be null.\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"primary\": {\n \"type\": \"boolean\",\n \"deprecationMessage\": \"Use the primaryKey field instead.\"\n },\n \"primaryKey\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"If this field is a primary key.\"\n },\n \"references\": {\n \"type\": \"string\",\n \"description\": \"The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship.\",\n \"examples\": [\n \"orders.order_id\",\n \"model.nested_field.field\"\n ]\n },\n \"unique\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if the value must be unique within the model.\"\n },\n \"enum\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"uniqueItems\": true,\n \"description\": \"A value must be equal to one of the elements in this array value. Only evaluated if the value is not null.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must greater than, or equal to, the value of this. Only applies to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must less than, or equal to, the value of this. Only applies to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"A specific format the value must comply with (e.g., 'email', 'uri', 'uuid').\",\n \"examples\": [\n \"email\",\n \"uri\",\n \"uuid\"\n ]\n },\n \"precision\": {\n \"type\": \"number\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"number\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression the value must match. Only applies to string types.\",\n \"examples\": [\n \"^[a-zA-Z0-9_-]+$\"\n ]\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\n \"sensitive\",\n \"restricted\",\n \"internal\",\n \"public\"\n ]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"lineage\": {\n \"$ref\": \"#/$defs/Lineage\"\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for field configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroType\": {\n \"type\": \"string\",\n \"description\": \"Specify the field type to use when exporting the data model to Apache Avro.\"\n },\n \"avroLogicalType\": {\n \"type\": \"string\",\n \"description\": \"Specify the logical field type to use when exporting the data model to Apache Avro.\"\n },\n \"bigqueryType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)`.\"\n },\n \"snowflakeType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Snowflake table, e.g., `TIMESTAMP_LTZ`.\"\n },\n \"redshiftType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Redshift table, e.g., `SMALLINT`.\"\n },\n \"sqlserverType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a SQL Server table, e.g., `DATETIME2`.\"\n },\n \"databricksType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Databricks Unity Catalog table.\"\n },\n \"glueType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in an AWS Glue Data Catalog table.\"\n }\n }\n }\n }\n }\n },\n \"primaryKey\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"The compound primary key of the model.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"examples\": {\n \"type\": \"array\"\n },\n \"additionalFields\": {\n \"type\": \"boolean\",\n \"description\": \" Specify, if the model can have additional fields that are not defined in the contract. \",\n \"default\": false\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for model configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroNamespace\": {\n \"type\": \"string\",\n \"description\": \"The namespace to use when importing and exporting the data model from / to Apache Avro.\"\n }\n }\n }\n }\n }\n },\n \"definitions\": {\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"type\": \"object\",\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9/_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Definition\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\",\n \"deprecationMessage\": \"This field is deprecated. Encode the domain into the ID using slashes.\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\",\n \"deprecationMessage\": \"This field is deprecated. Encode the name into the ID using slashes.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"Example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n }\n },\n \"servicelevels\": {\n \"type\": \"object\",\n \"description\": \"Specifies the service level agreements for the provided data, including availability, data retention policies, latency requirements, data freshness, update frequency, support availability, and backup policies.\",\n \"properties\": {\n \"availability\": {\n \"type\": \"object\",\n \"description\": \"Availability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the availability service level.\",\n \"example\": \"The server is available during support hours\"\n },\n \"percentage\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the guaranteed uptime in percent (e.g., `99.9%`)\",\n \"pattern\": \"^\\\\d+(\\\\.\\\\d+)?%$\",\n \"example\": \"99.9%\"\n }\n }\n },\n \"retention\": {\n \"type\": \"object\",\n \"description\": \"Retention covers the period how long data will be available.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the retention service level.\",\n \"example\": \"Data is retained for one year.\"\n },\n \"period\": {\n \"type\": \"string\",\n \"description\": \"An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`).\",\n \"example\": \"P1Y\"\n },\n \"unlimited\": {\n \"type\": \"boolean\",\n \"description\": \"An optional indicator that data is kept forever.\",\n \"example\": false\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the period refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"latency\": {\n \"type\": \"object\",\n \"description\": \"Latency refers to the maximum amount of time from the source to its destination.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the latency service level.\",\n \"example\": \"Data is available within 25 hours after the order was placed.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`).\",\n \"example\": \"25h\"\n },\n \"sourceTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp when the data was provided at the source.\",\n \"example\": \"orders.order_timestamp\"\n },\n \"processedTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract.\",\n \"example\": \"orders.processed_timestamp\"\n }\n }\n },\n \"freshness\": {\n \"type\": \"object\",\n \"description\": \"The maximum age of the youngest row in a table.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the freshness service level.\",\n \"example\": \"The age of the youngest row in a table is within 25 hours.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g., `PT24H`).\",\n \"example\": \"25h\"\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the threshold refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"frequency\": {\n \"type\": \"object\",\n \"description\": \"Frequency describes how often data is updated.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the frequency service level.\",\n \"example\": \"Data is delivered once a day.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"batch\",\n \"micro-batching\",\n \"streaming\",\n \"manual\"\n ],\n \"description\": \"The method of data processing.\",\n \"example\": \"batch\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`.\",\n \"example\": \"daily\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * *\"\n }\n }\n },\n \"support\": {\n \"type\": \"object\",\n \"description\": \"Support describes the times when support will be available for contact.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the support service level.\",\n \"example\": \"The data is available during typical business hours at headquarters.\"\n },\n \"time\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`.\",\n \"example\": \"9am to 5pm in EST on business days\"\n },\n \"responseTime\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with.\",\n \"example\": \"24 hours\"\n }\n }\n },\n \"backup\": {\n \"type\": \"object\",\n \"description\": \"Backup specifies details about data backup procedures.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the backup service level.\",\n \"example\": \"Data is backed up once a week, every Sunday at 0:00 UTC.\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"An optional interval that defines how often data will be backed up, e.g., `daily`.\",\n \"example\": \"weekly\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"An optional cron expression when data will be backed up, e.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * 0\"\n },\n \"recoveryTime\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours).\",\n \"example\": \"24 hours\"\n },\n \"recoveryPoint\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours).\",\n \"example\": \"1 week\"\n }\n }\n }\n }\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"description\": \"Tags to facilitate searching and filtering.\",\n \"examples\": [\n \"databricks\",\n \"pii\",\n \"sensitive\"\n ]\n },\n \"description\": \"Tags to facilitate searching and filtering.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ],\n \"$defs\": {\n \"FieldType\": {\n \"type\": \"string\",\n \"title\": \"FieldType\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\",\n \"decimal\",\n \"numeric\",\n \"int\",\n \"integer\",\n \"long\",\n \"bigint\",\n \"float\",\n \"double\",\n \"string\",\n \"text\",\n \"varchar\",\n \"boolean\",\n \"timestamp\",\n \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"time\",\n \"array\",\n \"map\",\n \"object\",\n \"record\",\n \"struct\",\n \"bytes\",\n \"variant\",\n \"json\",\n \"null\"\n ]\n },\n \"BaseServer\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the servers.\"\n },\n \"environment\": {\n \"type\": \"string\",\n \"description\": \"The environment in which the servers are running. Examples: prod, sit, stg.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the data product technology that implements the data contract.\",\n \"examples\": [\n \"azure\",\n \"bigquery\",\n \"BigQuery\",\n \"clickhouse\",\n \"databricks\",\n \"dataframe\",\n \"glue\",\n \"kafka\",\n \"kinesis\",\n \"local\",\n \"oracle\",\n \"postgres\",\n \"pubsub\",\n \"redshift\",\n \"sftp\",\n \"sqlserver\",\n \"snowflake\",\n \"s3\",\n \"trino\"\n ]\n },\n \"roles\": {\n \"description\": \" An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The name of the role.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the role and what access the role provides.\"\n }\n },\n \"required\": [\n \"name\"\n ]\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\"\n ]\n },\n \"BigQueryServer\": {\n \"type\": \"object\",\n \"title\": \"BigQueryServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"The GCP dataset name.\"\n }\n },\n \"required\": [\n \"project\",\n \"dataset\"\n ]\n },\n \"S3Server\": {\n \"type\": \"object\",\n \"title\": \"S3Server\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"S3 URL, starting with `s3://`\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}/*.json\"\n ]\n },\n \"endpointUrl\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The server endpoint for S3-compatible servers.\",\n \"examples\": [\n \"https://minio.example.com\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"SftpServer\": {\n \"type\": \"object\",\n \"title\": \"SftpServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"pattern\": \"^sftp://.*\",\n \"description\": \"SFTP URL, starting with `sftp://`\",\n \"examples\": [\n \"sftp://123.123.12.123/{model}/*.json\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"RedshiftServer\": {\n \"type\": \"object\",\n \"title\": \"RedshiftServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the host name.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"clusterIdentifier\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's identifier.\",\n \"examples\": [\n \"redshift-prod-eu\",\n \"analytics-cluster\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"An optional string describing the cluster's port.\",\n \"examples\": [\n 5439\n ]\n },\n \"endpoint\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's endpoint.\",\n \"examples\": [\n \"analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"AzureServer\": {\n \"type\": \"object\",\n \"title\": \"AzureServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"Path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs. Recommended pattern is 'abfss:///'\",\n \"examples\": [\n \"abfss://my_container_name/path\",\n \"abfss://my_container_name/path/*.json\",\n \"az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet\",\n \"abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\",\n \"format\"\n ]\n },\n \"SqlserverServer\": {\n \"type\": \"object\",\n \"title\": \"SqlserverServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\",\n \"default\": 1433,\n \"examples\": [\n 1433\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"database\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"dbo\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"database\",\n \"schema\"\n ]\n },\n \"SnowflakeServer\": {\n \"type\": \"object\",\n \"title\": \"SnowflakeServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"DatabricksServer\": {\n \"type\": \"object\",\n \"title\": \"DatabricksServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Databricks host\",\n \"examples\": [\n \"dbc-abcdefgh-1234.cloud.databricks.com\"\n ]\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the Hive or Unity catalog\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The schema name in the catalog\"\n }\n },\n \"required\": [\n \"catalog\",\n \"schema\"\n ]\n },\n \"DataframeServer\": {\n \"type\": \"object\",\n \"title\": \"DataframeServer\",\n \"required\": [\n \"type\"\n ]\n },\n \"GlueServer\": {\n \"type\": \"object\",\n \"title\": \"GlueServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue account\",\n \"examples\": [\n \"1234-5678-9012\"\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue database name\",\n \"examples\": [\n \"my_database\"\n ]\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The AWS S3 path. Must be in the form of a URL.\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the files\",\n \"examples\": [\n \"parquet\",\n \"csv\",\n \"json\",\n \"delta\"\n ]\n }\n },\n \"required\": [\n \"account\",\n \"database\"\n ]\n },\n \"PostgresServer\": {\n \"type\": \"object\",\n \"title\": \"PostgresServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"public\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\",\n \"schema\"\n ]\n },\n \"OracleServer\": {\n \"type\": \"object\",\n \"title\": \"OracleServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the oracle server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the oracle server.\",\n \"examples\": [\n 1523\n ]\n },\n \"serviceName\": {\n \"type\": \"string\",\n \"description\": \"The name of the service.\",\n \"examples\": [\n \"service\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"serviceName\"\n ]\n },\n \"KafkaServer\": {\n \"type\": \"object\",\n \"title\": \"KafkaServer\",\n \"description\": \"Kafka Server\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The bootstrap server of the kafka cluster.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the message. Examples: json, avro, protobuf.\",\n \"default\": \"json\"\n }\n },\n \"required\": [\n \"host\",\n \"topic\"\n ]\n },\n \"PubSubServer\": {\n \"type\": \"object\",\n \"title\": \"PubSubServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n }\n },\n \"required\": [\n \"project\",\n \"topic\"\n ]\n },\n \"KinesisDataStreamsServer\": {\n \"type\": \"object\",\n \"title\": \"KinesisDataStreamsServer\",\n \"description\": \"Kinesis Data Streams Server\",\n \"properties\": {\n \"stream\": {\n \"type\": \"string\",\n \"description\": \"The name of the Kinesis data stream.\"\n },\n \"region\": {\n \"type\": \"string\",\n \"description\": \"AWS region.\",\n \"examples\": [\n \"eu-west-1\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the record\",\n \"examples\": [\n \"json\",\n \"avro\",\n \"protobuf\"\n ]\n }\n },\n \"required\": [\n \"stream\"\n ]\n },\n \"TrinoServer\": {\n \"type\": \"object\",\n \"title\": \"TrinoServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Trino host URL.\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The Trino port.\"\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the catalog.\",\n \"examples\": [\n \"hive\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"my_schema\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"catalog\",\n \"schema\"\n ]\n },\n \"ClickhouseServer\": {\n \"type\": \"object\",\n \"title\": \"ClickhouseServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\"\n ]\n },\n \"LocalServer\": {\n \"type\": \"object\",\n \"title\": \"LocalServer\",\n \"properties\": {\n \"path\": {\n \"type\": \"string\",\n \"description\": \"The relative or absolute path to the data file(s).\",\n \"examples\": [\n \"./folder/data.parquet\",\n \"./folder/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the file(s)\",\n \"examples\": [\n \"json\",\n \"parquet\",\n \"delta\",\n \"csv\"\n ]\n }\n },\n \"required\": [\n \"path\",\n \"format\"\n ]\n },\n \"Quality\": {\n \"allOf\": [\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of quality check\",\n \"enum\": [\n \"text\",\n \"library\",\n \"sql\",\n \"custom\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n }\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"text\"\n }\n }\n },\n \"then\": {\n \"required\": [\n \"description\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sql\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"query\": {\n \"type\": \"string\",\n \"description\": \"A SQL query that returns a single number to compare with the threshold.\"\n },\n \"dialect\": {\n \"type\": \"string\",\n \"description\": \"The SQL dialect that is used for the query. Should be compatible to the server.type.\",\n \"examples\": [\n \"athena\",\n \"bigquery\",\n \"redshift\",\n \"snowflake\",\n \"trino\",\n \"postgres\",\n \"oracle\"\n ]\n },\n \"mustBe\": {\n \"type\": \"number\"\n },\n \"mustNotBe\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterOrEqualTo\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterThanOrEqualTo\": {\n \"type\": \"number\",\n \"deprecated\": true\n },\n \"mustBeLessThan\": {\n \"type\": \"number\"\n },\n \"mustBeLessThanOrEqualTo\": {\n \"type\": \"number\",\n \"deprecated\": true\n },\n \"mustBeLessOrEqualTo\": {\n \"type\": \"number\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n }\n },\n \"required\": [\n \"query\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"library\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"metric\": {\n \"type\": \"string\",\n \"description\": \"The DataQualityLibrary metric to use for the quality check.\",\n \"examples\": [\"nullValues\", \"missingValues\", \"invalidValues\", \"duplicateValues\", \"rowCount\"]\n },\n \"rule\": {\n \"type\": \"string\",\n \"deprecated\": true,\n \"description\": \"Deprecated. Use metric instead\"\n },\n \"arguments\": {\n \"type\": \"object\",\n \"description\": \"Additional metric-specific parameters for the quality check.\",\n \"additionalProperties\": {\n \"type\": [\"string\", \"number\", \"boolean\", \"array\", \"object\"]\n }\n },\n \"mustBe\": {\n \"description\": \"Must be equal to the value to be valid. When using numbers, it is equivalent to '='.\"\n },\n \"mustNotBe\": {\n \"description\": \"Must not be equal to the value to be valid. When using numbers, it is equivalent to '!='.\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than the value to be valid. It is equivalent to '>'.\"\n },\n \"mustBeGreaterOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than or equal to the value to be valid. It is equivalent to '>='.\"\n },\n \"mustBeLessThan\": {\n \"type\": \"number\",\n \"description\": \"Must be less than the value to be valid. It is equivalent to '<'.\"\n },\n \"mustBeLessOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be less than or equal to the value to be valid. It is equivalent to '<='.\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must not be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n }\n },\n \"required\": [\n \"metric\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"custom\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n },\n \"engine\": {\n \"type\": \"string\",\n \"examples\": [\n \"soda\",\n \"great-expectations\"\n ],\n \"description\": \"The engine used for custom quality checks.\"\n },\n \"implementation\": {\n \"type\": [\n \"object\",\n \"array\",\n \"string\"\n ],\n \"description\": \"Engine-specific quality checks and expectations.\"\n }\n },\n \"required\": [\n \"engine\"\n ]\n }\n }\n ]\n },\n \"Lineage\": {\n \"type\": \"object\",\n \"properties\": {\n \"inputFields\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"namespace\": {\n \"type\": \"string\",\n \"description\": \"The input dataset namespace\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The input dataset name\"\n },\n \"field\": {\n \"type\": \"string\",\n \"description\": \"The input field\"\n },\n \"transformations\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"description\": \"The type of the transformation. Allowed values are: DIRECT, INDIRECT\",\n \"type\": \"string\"\n },\n \"subtype\": {\n \"type\": \"string\",\n \"description\": \"The subtype of the transformation\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\"\n },\n \"masking\": {\n \"type\": \"boolean\",\n \"description\": \"is transformation masking the data or not\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"additionalProperties\": true\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"namespace\",\n \"name\",\n \"field\"\n ]\n }\n },\n \"transformationDescription\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\",\n \"deprecated\": true\n },\n \"transformationType\": {\n \"type\": \"string\",\n \"description\": \"IDENTITY|MASKED reflects a clearly defined behavior. IDENTITY: exact same as input; MASKED: no original data available (like a hash of PII for example)\",\n \"deprecated\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"inputFields\"\n ]\n }\n }\n}\n" }, { "path": "definition.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"properties\": {\n \"id\": {\n \"type\": \"string\",\n \"description\": \"A unique identifier for this definition. Encode the domain into the ID, separated by slashes.\",\n \"examples\": [\n \"checkout/order_id\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n}\n" }, { "path": "diagrams/automation.drawio", "content": "\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n" }, { "path": "diagrams/datacontract.drawio", "content": "\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n" }, { "path": "diagrams/favicon.drawio", "content": "" }, { "path": "examples/covid-cases/datacontract.html", "content": "\n\n\n Data Contract\n \n \n \n \n\n\n\n
\n\n \n\n
\n\n
\n
\n
\n
\n

\n Data Contract

\n
\n covid_cases\n
\n
\n \n
\n
\n
\n \n \n \n \n Show YAML\n \n
\n
\n
\n\n
\n
\n
\n
\n

Info

\n

Information about the data contract

\n
\n
\n\n
\n\n \n\n
\n
\n
Title
\n
COVID-19 cases
\n
\n\n
\n
Version
\n
0.0.1
\n
\n\n \n\n \n
\n
Description
\n
\n Johns Hopkins University Consolidated data on COVID-19 cases, sourced from Enigma\n
\n
\n \n\n \n\n \n
\n
links
\n
{'blog': 'https://aws.amazon.com/blogs/big-data/a-public-data-lake-for-analysis-of-covid-19-data/', 'data-explorer': 'https://dj2taa9i652rf.cloudfront.net/', 'data': 'https://covid19-lake.s3.us-east-2.amazonaws.com/enigma-jhu/json/part-00000-adec1cd2-96df-4c6b-a5f2-780f092951ba-c000.json'}
\n
\n \n\n \n\n
\n
\n
\n
\n\n\n \n
\n
\n

Servers

\n

Servers of the data contract

\n
\n\n
    \n\n \n
  • \n
    \n
    \n
    \n
    Server
    \n
    s3-json
    \n
    \n
    \n
    \n\n \n\n \n
    \n
    \n
    \n
    Type
    \n
    s3
    \n
    \n
    \n
    \n \n\n \n\n \n\n \n
    \n
    \n
    \n
    Location
    \n
    s3://covid19-lake/enigma-jhu/json/*.json
    \n
    \n
    \n
    \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n
    \n
    \n
    \n
    Format
    \n
    json
    \n
    \n
    \n
    \n \n\n \n
    \n
    \n
    \n
    Delimiter
    \n
    new_line
    \n
    \n
    \n
    \n \n\n \n\n \n\n
    • \n \n\n
\n\n
\n \n\n\n \n\n\n
\n
\n
\n

\n Data Model\n

\n

The logical data model

\n
\n
\n\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n
\n covid_cases\n table\n
the number of confirmed covid cases reported for a specified region, with location and county/province/country information.
\n
\n\n
\n \n fips\n
\n 		\n \n string\n \n \n \n
state and county two digits code
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n admin2\n
\n 		\n \n string\n \n \n \n
county name
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n province_state\n
\n 		\n \n string\n \n \n \n
province name or state name
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n country_region\n
\n 		\n \n string\n \n \n \n
country name or region name
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n last_update\n
\n 		\n \n timestamp_ntz\n \n \n \n
last update timestamp
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n latitude\n
\n 		\n \n double\n \n \n \n
location (latitude)
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n longitude\n
\n 		\n \n double\n \n \n \n
location (longitude)
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n confirmed\n
\n 		\n \n int\n \n \n \n
number of confirmed cases
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n combined_key\n
\n 		\n \n string\n \n \n \n
county name+state name+country name
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n
\n
\n
\n \n
\n\n \n\n \n\n \n\n \n
\n
\n

\n Quality\n

\n

\n SodaCL\n

\n
\n
\n
\n
\n 
checks for covid_cases:\n- freshness(last_update::datetime) < 5000d\n- row_count > 1000\n
\n
\n
\n
\n
\n \n\n
\n
\n\n
\n Created at 27 Jun 2024 14:50:11 UTC with Data Contract CLI v0.10.8\n
\n\n
\n
\n\n \n
\n\n
\n
\n
\n
\n \n
\n
\n
\n\n

Data Contract YAML

\n
\n 
dataContractSpecification: 0.9.3\nid: covid_cases\ninfo:\n  title: COVID-19 cases\n  version: 0.0.1\n  description: Johns Hopkins University Consolidated data on COVID-19 cases, sourced\n    from Enigma\n  links:\n    blog: https://aws.amazon.com/blogs/big-data/a-public-data-lake-for-analysis-of-covid-19-data/\n    data-explorer: https://dj2taa9i652rf.cloudfront.net/\n    data: https://covid19-lake.s3.us-east-2.amazonaws.com/enigma-jhu/json/part-00000-adec1cd2-96df-4c6b-a5f2-780f092951ba-c000.json\nservers:\n  s3-json:\n    type: s3\n    format: json\n    delimiter: new_line\n    location: s3://covid19-lake/enigma-jhu/json/*.json\nmodels:\n  covid_cases:\n    description: the number of confirmed covid cases reported for a specified region,\n      with location and county/province/country information.\n    type: table\n    fields:\n      fips:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: state and county two digits code\n      admin2:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: county name\n      province_state:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: province name or state name\n      country_region:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: country name or region name\n      last_update:\n        type: timestamp_ntz\n        required: false\n        primary: false\n        unique: false\n        description: last update timestamp\n      latitude:\n        type: double\n        required: false\n        primary: false\n        unique: false\n        description: location (latitude)\n      longitude:\n        type: double\n        required: false\n        primary: false\n        unique: false\n        description: location (longitude)\n      confirmed:\n        type: int\n        required: false\n        primary: false\n        unique: false\n        description: number of confirmed cases\n      combined_key:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: county name+state name+country name\nquality:\n  type: SodaCL\n  specification:\n    checks for covid_cases:\n    - freshness(last_update::datetime) < 5000d\n    - row_count > 1000\n
\n
\n
\n \n
\n
\n
\n
\n \n\n\n \n\n
\n\n\n" }, { "path": "examples/covid-cases/datacontract.yaml", "content": "dataContractSpecification: 0.9.3\nid: covid_cases\ninfo:\n title: COVID-19 cases\n description: Johns Hopkins University Consolidated data on COVID-19 cases, sourced from Enigma\n version: \"0.0.1\"\n links:\n blog: https://aws.amazon.com/blogs/big-data/a-public-data-lake-for-analysis-of-covid-19-data/\n data-explorer: https://dj2taa9i652rf.cloudfront.net/\n data: https://covid19-lake.s3.us-east-2.amazonaws.com/enigma-jhu/json/part-00000-adec1cd2-96df-4c6b-a5f2-780f092951ba-c000.json\nservers:\n s3-json:\n type: s3\n location: s3://covid19-lake/enigma-jhu/json/*.json\n format: json\n delimiter: new_line\nmodels:\n covid_cases:\n description: the number of confirmed covid cases reported for a specified region, with location and county/province/country information.\n fields:\n fips:\n type: string\n description: state and county two digits code\n admin2:\n type: string\n description: county name\n province_state:\n type: string\n description: province name or state name\n country_region:\n type: string\n description: country name or region name\n last_update:\n type: timestamp_ntz\n description: last update timestamp\n latitude:\n type: double\n description: location (latitude)\n longitude:\n type: double\n description: location (longitude)\n confirmed:\n type: int\n description: number of confirmed cases\n combined_key:\n type: string\n description: county name+state name+country name\nquality:\n type: SodaCL\n specification:\n checks for covid_cases:\n - freshness(last_update::datetime) < 5000d # dataset is not updated anymore\n - row_count > 1000\n" }, { "path": "examples/datacontract.html", "content": "\n\n\n Data Contract\n \n \n \n \n\n\n\n
\n\n \n\n
\n\n
\n
\n
\n
\n

\n Data Contract

\n
\n covid_cases\n
\n
\n
\n \n \n \n \n Show YAML\n \n
\n
\n
\n\n
\n
\n
\n
\n

Info

\n

Information about the data contract

\n
\n
\n\n
\n\n
\n
\n
Title
\n
COVID-19 cases
\n
\n\n
\n
Version
\n
0.0.1
\n
\n\n \n
\n
Description
\n
\n Johns Hopkins University Consolidated data on COVID-19 cases, sourced from Enigma\n
\n
\n \n\n \n\n \n\n
\n
\n
\n
\n\n\n \n
\n
\n

Servers

\n

Servers of the data contract

\n
\n\n
    \n\n \n
  • \n
    \n
    \n
    Server
    \n
    s3-json
    \n
    \n
    \n\n \n
    \n
    \n
    Type
    \n
    s3
    \n
    \n
    \n \n\n \n\n \n\n \n
    \n
    \n
    Location
    \n
    s3://covid19-lake/enigma-jhu/json/*.json
    \n
    \n
    \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n
    \n
    \n
    Format
    \n
    json
    \n
    \n
    \n \n\n \n
    \n
    \n
    Delimiter
    \n
    new_line
    \n
    \n
    \n \n\n
    • \n \n\n
\n\n
\n \n\n\n \n\n\n
\n
\n
\n

\n Data Model\n

\n

The logical data model

\n
\n
\n\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n\n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n \n \n \n \n\n \n \n\n \n
\n covid_cases\n table\n
the number of confirmed covid cases reported for a specified region, with location and county/province/country information.
\n
\n
\n fips\n \n
\n 		\n \n \n \n \n string\n \n \n
state and county two digits code
\n \n
\n
\n admin2\n \n
\n 		\n \n \n \n \n string\n \n \n
county name
\n \n
\n
\n province_state\n \n
\n 		\n \n \n \n \n string\n \n \n
province name or state name
\n \n
\n
\n country_region\n \n
\n 		\n \n \n \n \n string\n \n \n
country name or region name
\n \n
\n
\n last_update\n \n
\n 		\n \n \n \n \n timestamp_ntz\n \n \n
last update timestamp
\n \n
\n
\n latitude\n \n
\n 		\n \n \n \n \n double\n \n \n
location (latitude)
\n \n
\n
\n longitude\n \n
\n 		\n \n \n \n \n double\n \n \n
location (longitude)
\n \n
\n
\n confirmed\n \n
\n 		\n \n \n \n \n int\n \n \n
number of confirmed cases
\n \n
\n
\n combined_key\n \n
\n 		\n \n \n \n \n string\n \n \n
county name+state name+country name
\n \n
\n
\n
\n
\n
\n \n\n
\n\n \n\n \n\n \n
\n
\n

\n Quality\n

\n

\n SodaCL\n

\n\n\n
\n
\n
\n
\n 
checks for covid_cases:\n- freshness(last_update::datetime) < 5000d\n- row_count > 1000\n
\n
\n
\n
\n
\n \n\n
\n\n
\n\n
\n Created at 29 Apr 2024 19:30:08 UTC with Data Contract CLI v0.10.1\n
\n\n
\n
\n\n \n
\n\n
\n
\n
\n
\n \n
\n
\n
\n\n

Data Contract YAML

\n
\n 
dataContractSpecification: 0.9.3\nid: covid_cases\ninfo:\n  title: COVID-19 cases\n  version: 0.0.1\n  description: Johns Hopkins University Consolidated data on COVID-19 cases, sourced\n    from Enigma\nservers:\n  s3-json:\n    type: s3\n    format: json\n    delimiter: new_line\n    location: s3://covid19-lake/enigma-jhu/json/*.json\nmodels:\n  covid_cases:\n    description: the number of confirmed covid cases reported for a specified region,\n      with location and county/province/country information.\n    type: table\n    fields:\n      fips:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: state and county two digits code\n      admin2:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: county name\n      province_state:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: province name or state name\n      country_region:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: country name or region name\n      last_update:\n        type: timestamp_ntz\n        required: false\n        primary: false\n        unique: false\n        description: last update timestamp\n      latitude:\n        type: double\n        required: false\n        primary: false\n        unique: false\n        description: location (latitude)\n      longitude:\n        type: double\n        required: false\n        primary: false\n        unique: false\n        description: location (longitude)\n      confirmed:\n        type: int\n        required: false\n        primary: false\n        unique: false\n        description: number of confirmed cases\n      combined_key:\n        type: string\n        required: false\n        primary: false\n        unique: false\n        description: county name+state name+country name\nquality:\n  type: SodaCL\n  specification:\n    checks for covid_cases:\n    - freshness(last_update::datetime) < 5000d\n    - row_count > 1000\n
\n
\n
\n \n
\n
\n
\n
\n \n\n\n \n\n
\n\n\n" }, { "path": "examples/generate-catalog", "content": "datacontract catalog --files \"**/*.yaml\" --output \".\"\n" }, { "path": "examples/index.html", "content": "\n\n\n Data Contract\n \n \n \n \n\n\n\n
\n\n \n\n
\n\n
\n
\n
\n
\n

\n Data Contract Catalog

\n
\n
\n\n\n
\n\n
\n\n
\n\n
\n\n \n\n
\n

Filters

\n
\n
\n
\n
\n
\n \n \n \n
\n \n
\n
\n
\n
\n
\n \n\n\n \n\n \n\n\n
\n No results found.\n
\n\n
\n\n
\n\n
\n\n
\n Created at 27 Jun 2024 14:50:13 UTC with Data Contract CLI v0.10.8\n
\n\n
\n
\n\n \n
\n\n
\n
\n
\n
\n \n
\n
\n
\n\n

Data Contract YAML

\n
\n
\n
\n
\n \n
\n
\n
\n
\n \n\n\n \n\n
\n\n\n\n\n" }, { "path": "examples/muellimperium/data.csv", "content": "Pluto,residual_waste,2021-01-09\nPluto,bio_waste,2021-01-02\nPluto,paper,2021-01-11\nPluto,plastic,2021-01-12\nPluto,bulky_waste,2021-02-04\nEarth,residual_waste,2021-01-14\nEarth,bio_waste,2021-01-08\nEarth,paper,2021-01-12\nEarth,plastic,2021-01-27\nEarth,bulky_waste,2021-02-03\n" }, { "path": "examples/muellimperium/datacontract.html", "content": "\n\n\n Data Contract\n \n \n \n \n\n\n\n
\n\n \n\n
\n\n
\n
\n
\n
\n

\n Data Contract

\n
\n muellimperium-exchange-format\n
\n
\n \n
\n
\n
\n \n \n \n \n Show YAML\n \n
\n
\n
\n\n
\n
\n
\n
\n

Info

\n

Information about the data contract

\n
\n
\n\n
\n\n \n\n
\n
\n
Title
\n
Muellimperium Exchange Format
\n
\n\n
\n
Version
\n
0.0.1
\n
\n\n \n\n \n
\n
Description
\n
\n The Muellimperium Exchange Format is a data contract for exchanging data between the Muellimperium and its partners.\n\n
\n
\n \n\n \n
\n
Owner
\n
\n Emperor of the Muellimperium\n
\n
\n \n\n \n
\n
contract
\n
{'name': 'The Emperor', 'email': 'the-emperor@muellimperium.com'}
\n
\n \n\n \n\n
\n
\n
\n
\n\n\n \n
\n
\n

Servers

\n

Servers of the data contract

\n
\n\n
    \n\n \n
  • \n
    \n
    \n
    \n
    Server
    \n
    exchange
    \n
    \n
    \n
    \n\n \n\n \n
    \n
    \n
    \n
    Type
    \n
    local
    \n
    \n
    \n
    \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n
    \n
    \n
    \n
    Path
    \n
    data.csv
    \n
    \n
    \n
    \n \n\n \n
    \n
    \n
    \n
    Format
    \n
    csv
    \n
    \n
    \n
    \n \n\n \n\n \n\n \n\n
    • \n \n\n
\n\n
\n \n\n\n \n\n\n
\n
\n
\n

\n Data Model\n

\n

The logical data model

\n
\n
\n\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n
\n garbage_collection\n table\n
None
\n
\n\n
\n \n location\n
\n 		\n \n text\n \n \n \n
The location where the garbage is collected.
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n garbage_type\n
\n 		\n \n text\n \n \n \n
The type of garbage that is collected.
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n collection_date\n
\n 		\n \n date\n \n \n \n
The date when the garbage is collected.
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n
\n
\n
\n \n
\n\n \n\n \n
\n
\n

Examples

\n

Examples for models in the data contract

\n
\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n
\n garbage_collection\n json\n
None
\n
\n 
[{'location': 'Musterstadt', 'garbage_type': 'paper', 'collection_date': '2022-01-01'}, {'location': 'Musterstadt', 'garbage_type': 'plastic', 'collection_date': '2022-01-02'}, {'location': 'Musterstadt', 'garbage_type': 'residual_waste', 'collection_date': '2022-01-03'}]
\n
\n
\n
\n
\n
\n \n
\n \n\n \n\n \n\n
\n
\n\n
\n Created at 27 Jun 2024 14:50:12 UTC with Data Contract CLI v0.10.8\n
\n\n
\n
\n\n \n
\n\n
\n
\n
\n
\n \n
\n
\n
\n\n

Data Contract YAML

\n
\n 
dataContractSpecification: 0.9.3\nid: muellimperium-exchange-format\ninfo:\n  title: Muellimperium Exchange Format\n  version: 0.0.1\n  description: 'The Muellimperium Exchange Format is a data contract for exchanging\n    data between the Muellimperium and its partners.\n\n    '\n  owner: Emperor of the Muellimperium\n  contract:\n    name: The Emperor\n    email: the-emperor@muellimperium.com\nservers:\n  exchange:\n    type: local\n    format: csv\n    path: data.csv\nmodels:\n  garbage_collection:\n    type: table\n    fields:\n      location:\n        type: text\n        required: true\n        primary: false\n        unique: false\n        description: The location where the garbage is collected.\n      garbage_type:\n        type: text\n        required: true\n        primary: false\n        unique: false\n        description: The type of garbage that is collected.\n        enum:\n        - paper\n        - plastic\n        - residual_waste\n        - bio_waste\n        - bulky_waste\n        - hazardous_waste\n      collection_date:\n        type: date\n        required: true\n        primary: false\n        unique: false\n        description: The date when the garbage is collected.\nexamples:\n- type: json\n  model: garbage_collection\n  data:\n  - location: Musterstadt\n    garbage_type: paper\n    collection_date: '2022-01-01'\n  - location: Musterstadt\n    garbage_type: plastic\n    collection_date: '2022-01-02'\n  - location: Musterstadt\n    garbage_type: residual_waste\n    collection_date: '2022-01-03'\n
\n
\n
\n \n
\n
\n
\n
\n \n\n\n \n\n
\n\n\n" }, { "path": "examples/muellimperium/datacontract.yaml", "content": "dataContractSpecification: 0.9.3\nid: muellimperium-exchange-format\ninfo:\n title: Muellimperium Exchange Format\n version: 0.0.1\n description: |\n The Muellimperium Exchange Format is a data contract for exchanging data between the Muellimperium and its partners.\n owner: Emperor of the Muellimperium\n contract:\n name: The Emperor\n email: the-emperor@muellimperium.com\nservers:\n exchange:\n type: local\n path: data.csv\n format: csv\nmodels:\n garbage_collection:\n type: table\n fields:\n location:\n type: text\n required: true\n description: The location where the garbage is collected.\n garbage_type:\n type: text\n required: true\n description: The type of garbage that is collected.\n enum:\n - paper\n - plastic\n - residual_waste\n - bio_waste\n - bulky_waste\n - hazardous_waste\n collection_date:\n type: date\n required: true\n description: The date when the garbage is collected.\nexamples:\n - model: garbage_collection\n type: json\n data:\n - location: \"Musterstadt\"\n garbage_type: \"paper\"\n collection_date: \"2022-01-01\"\n - location: \"Musterstadt\"\n garbage_type: \"plastic\"\n collection_date: \"2022-01-02\"\n - location: \"Musterstadt\"\n garbage_type: \"residual_waste\"\n collection_date: \"2022-01-03\"\n" }, { "path": "examples/orders-latest/datacontract.html", "content": "\n\n\n Data Contract\n \n \n \n \n\n\n\n
\n\n \n\n
\n\n
\n
\n
\n
\n

\n Data Contract

\n
\n urn:orders-latest\n
\n
\n \n
\n \n \n \n \n \n \n checkout\n \n \n \n orders\n \n \n \n s3\n \n \n
\n \n
\n
\n
\n \n \n \n \n Show YAML\n \n
\n
\n
\n\n
\n
\n
\n
\n

Info

\n

Information about the data contract

\n
\n
\n\n
\n\n \n
\n \n \n
\n \n
\n
datacontractCli
\n \n \n
\n \n\n
\n
\n
Title
\n
Orders Latest
\n
\n\n
\n
Version
\n
1.0.0
\n
\n\n \n\n \n
\n
Description
\n
\n Successful customer orders in the webshop. \nAll orders since 2020-01-01. \nOrders with their line items are in their current state (no history included).\n\n
\n
\n \n\n \n
\n
Owner
\n
\n Checkout Team\n
\n
\n \n\n \n
\n
slackChannel
\n
#checkout
\n
\n \n\n \n
\n
Contact
\n
\n \n John Doe (Data Product Owner)\n \n \n \n \n \n
\n
\n \n\n
\n
\n
\n
\n\n\n \n
\n
\n

Servers

\n

Servers of the data contract

\n
\n\n
    \n\n \n
  • \n
    \n
    \n
    \n
    Server
    \n
    production
    \n
    \n
    \n
    \n\n \n
    \n
    \n
    \n
    Environment
    \n
    prod
    \n
    \n
    \n
    \n \n\n \n
    \n
    \n
    \n
    Type
    \n
    s3
    \n
    \n
    \n
    \n \n\n \n\n \n\n \n
    \n
    \n
    \n
    Location
    \n
    s3://datacontract-example-orders-latest/data/{model}/*.json
    \n
    \n
    \n
    \n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n
    \n
    \n
    \n
    Format
    \n
    json
    \n
    \n
    \n
    \n \n\n \n
    \n
    \n
    \n
    Delimiter
    \n
    new_line
    \n
    \n
    \n
    \n \n\n \n
    \n
    \n
    \n
    Description
    \n
    One folder per model. One file per day.
    \n
    \n
    \n
    \n \n\n \n\n
    • \n \n\n
\n\n
\n \n\n\n \n
\n
\n

Terms

\n

Terms and conditions of the data contract

\n
\n
\n\n
\n\n
\n
\n
Usage
\n
\n Data can be used for reports, analytics and machine learning use cases.\nOrder may be linked and joined by other tables\n\n
\n
\n\n
\n
Limitations
\n
\n Not suitable for real-time use cases.\nData may not be used to identify individual customers.\nMax data processing per day: 10 TiB\n\n
\n
\n\n \n
\n
Billing
\n
\n 5000 USD per month\n
\n
\n \n\n \n
\n
Notice Period
\n
\n P3M\n
\n
\n \n\n \n\n
\n
\n
\n
\n \n\n\n
\n
\n
\n

\n Data Model\n

\n

The logical data model

\n
\n
\n\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n
\n orders\n table\n
One record per order. Includes cancelled and deleted orders.
\n
\n\n
\n \n Order ID
\n \n order_id \n \n \n
\n 		\n \n text\n \n \n \n
An internal ID that identifies an order in the online shop.
\n \n\n \n
\n Example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n
\n \n\n
\n \n primary\n \n \n required\n \n \n unique\n \n \n format:uuid\n \n \n \n \n \n \n \n \n \n \n \n restricted\n \n \n PII\n \n \n \n
\n
\n\n
\n \n order_timestamp\n
\n 		\n \n timestamp\n \n \n \n
The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.
\n \n\n \n
\n Example: 2024-09-09T08:30:00Z\n
\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n order_total\n
\n 		\n \n long\n \n \n \n
Total amount the smallest monetary unit (e.g., cents).
\n \n\n \n
\n Example: 9999\n
\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n customer_id\n
\n 		\n \n text\n \n \n \n
Unique identifier for the customer.
\n \n\n \n\n
\n \n \n \n \n \n minLength:10\n \n \n maxLength:20\n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n customer_email_address\n
\n 		\n \n text\n \n \n \n
The email address, as entered by the customer. The email address was not verified.
\n \n\n \n\n
\n \n \n required\n \n \n \n format:email\n \n \n \n \n \n \n \n \n \n \n \n sensitive\n \n \n PII\n \n \n \n
\n
\n\n
\n \n processed_timestamp\n
\n 		\n \n timestamp\n \n \n \n
The timestamp when the record was processed by the data platform.
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n
\n
\n
\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n
\n line_items\n table\n
A single article that is part of an order.
\n
\n\n
\n \n lines_item_id\n
\n 		\n \n text\n \n \n \n
Primary key of the lines_item_id table
\n \n\n \n\n
\n \n primary\n \n \n required\n \n \n unique\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n Order ID
\n \n order_id \n \n \n
\n 		\n \n text\n \n \n \n
An internal ID that identifies an order in the online shop.
\n \n\n \n
\n Example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n
\n \n\n
\n \n \n \n \n format:uuid\n \n \n \n \n \n \n \n \n \n \n \n restricted\n \n \n PII\n \n \n \n
\n
\n\n
\n \n Stock Keeping Unit
\n \n sku \n \n \n
\n 		\n \n text\n \n \n \n
The purchased article number
\n \n\n \n
\n Example: 96385074\n
\n \n\n
\n \n \n \n \n \n \n \n pattern:^[A-Za-z0-9]{8,14}$\n \n \n \n \n \n \n \n \n \n \n \n \n wikipedia\n \n \n
\n
\n
\n
\n
\n
\n \n
\n\n \n
\n
\n

Definitions

\n

Domain specific definitions in the data contract

\n
\n\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n\n \n \n
\n order_id\n checkout\n
An internal ID that identifies an order in the online shop.
\n
\n
\n \n
Order ID
\n \n
order_id
\n
\n 		\n
\n text\n \n uuid\n \n
\n 		\n \n
\n Example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n
\n \n \n
\n Tags:\n \n orders\n \n
\n \n \n
\n \n \n \n \n \n \n \n \n \n \n restricted\n \n \n PII\n \n \n \n
\n
\n
\n
\n
\n
\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n\n \n \n
\n sku\n inventory\n
A Stock Keeping Unit (SKU) is an internal unique identifier for an article. \nIt is typically associated with an article's barcode, such as the EAN/GTIN.\n
\n
\n
\n \n
Stock Keeping Unit
\n \n
sku
\n
\n 		\n
\n text\n \n
\n 		\n \n
\n Example: 96385074\n
\n \n \n
\n Tags:\n \n inventory\n \n
\n \n \n
\n \n \n \n pattern:^[A-Za-z0-9]{8,14}$\n \n \n \n \n \n \n \n \n \n \n \n \n wikipedia\n \n \n
\n
\n
\n
\n
\n
\n \n
\n \n\n \n
\n
\n

Examples

\n

Examples for models in the data contract

\n
\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n
\n orders\n csv\n
An example list of order records.
\n
\n 
order_id,order_timestamp,order_total,customer_id,customer_email_address,processed_timestamp\n\"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\",\"2030-09-09T08:31:00Z\"\n\"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\",\"2030-09-09T08:31:00Z\"\n\"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\",\"2030-09-09T08:31:00Z\"\n\"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n\"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n\"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\",\"2030-09-09T08:31:00Z\"\n\"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\",\"2030-09-09T08:31:00Z\"\n\"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\",\"2030-09-09T08:31:00Z\"\n\"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\",\"2030-09-09T08:31:00Z\"\n\"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\",\"2030-09-09T08:31:00Z\"\n
\n
\n
\n
\n
\n
\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n
\n line_items\n csv\n
An example list of line items.
\n
\n 
lines_item_id,order_id,sku\n\"LI-1\",\"1001\",\"5901234123457\"\n\"LI-2\",\"1001\",\"4001234567890\"\n\"LI-3\",\"1002\",\"5901234123457\"\n\"LI-4\",\"1002\",\"2001234567893\"\n\"LI-5\",\"1003\",\"4001234567890\"\n\"LI-6\",\"1003\",\"5001234567892\"\n\"LI-7\",\"1004\",\"5901234123457\"\n\"LI-8\",\"1005\",\"2001234567893\"\n\"LI-9\",\"1005\",\"5001234567892\"\n\"LI-10\",\"1005\",\"6001234567891\"\n
\n
\n
\n
\n
\n
\n \n
\n \n\n \n
\n
\n

Service Levels

\n

Service levels of the data contract

\n
\n
\n
\n
\n \n
\n

\n Availability\n

\n \n
\n
Description
\n
\n The server is available during support hours\n
\n
\n \n \n
\n
Percentage
\n
\n 99.9%\n
\n
\n \n
\n \n \n
\n

\n Retention\n

\n \n
\n
Description
\n
\n Data is retained for one year\n
\n
\n \n \n
\n
Period
\n
\n P1Y\n
\n
\n \n \n
\n \n \n
\n

\n Latency\n

\n \n
\n
Description
\n
\n Data is available within 25 hours after the order was placed\n
\n
\n \n \n
\n
Threshold
\n
\n 25h\n
\n
\n \n \n
\n
Source Timestamp field
\n
\n orders.order_timestamp\n
\n
\n \n \n
\n
Processed Timestamp field
\n
\n orders.processed_timestamp\n
\n
\n \n
\n \n \n
\n

\n Freshness\n

\n \n
\n
Description
\n
\n The age of the youngest row in a table.\n
\n
\n \n \n
\n
Threshold
\n
\n 25h\n
\n
\n \n \n
\n
Timestamp field
\n
\n orders.order_timestamp\n
\n
\n \n
\n \n \n
\n

\n Frequency\n

\n \n
\n
Description
\n
\n Data is delivered once a day\n
\n
\n \n \n
\n
Type
\n
\n batch\n
\n
\n \n \n
\n
Interval
\n
\n daily\n
\n
\n \n \n
\n
Cron
\n
\n 0 0 * * *\n
\n
\n \n
\n \n \n
\n

\n Support\n

\n \n
\n
Description
\n
\n The data is available during typical business hours at headquarters\n
\n
\n \n \n
\n
Time
\n
\n 9am to 5pm in EST on business days\n
\n
\n \n \n
\n
Response Time
\n
\n 1h\n
\n
\n \n
\n \n \n
\n

\n Backup\n

\n \n
\n
Description
\n
\n Data is backed up once a week, every Sunday at 0:00 UTC.\n
\n
\n \n \n \n
\n
Cron
\n
\n 0 0 * * 0\n
\n
\n \n \n
\n
Recovery Time
\n
\n 24 hours\n
\n
\n \n \n
\n
Recovery Point
\n
\n 1 week\n
\n
\n \n
\n \n
\n
\n
\n
\n \n\n \n
\n
\n

\n Quality\n

\n

\n SodaCL\n

\n
\n
\n
\n
\n 
checks for orders:\n- row_count >= 5\n- duplicate_count(order_id) = 0\nchecks for line_items:\n- values in (order_id) must exist in orders (order_id)\n- row_count >= 5\n
\n
\n
\n
\n
\n \n\n
\n
\n\n
\n Created at 27 Jun 2024 14:50:10 UTC with Data Contract CLI v0.10.8\n
\n\n
\n
\n\n \n
\n\n
\n
\n
\n
\n \n
\n
\n
\n\n

Data Contract YAML

\n
\n 
dataContractSpecification: 0.9.3\nid: urn:orders-latest\ninfo:\n  title: Orders Latest\n  version: 1.0.0\n  description: \"Successful customer orders in the webshop. \\nAll orders since 2020-01-01.\\\n    \\ \\nOrders with their line items are in their current state (no history included).\\n\"\n  owner: Checkout Team\n  contact:\n    name: John Doe (Data Product Owner)\n    url: https://teams.microsoft.com/l/channel/example/checkout\n  slackChannel: '#checkout'\nservers:\n  production:\n    type: s3\n    description: One folder per model. One file per day.\n    environment: prod\n    format: json\n    delimiter: new_line\n    location: s3://datacontract-example-orders-latest/data/{model}/*.json\nterms:\n  usage: 'Data can be used for reports, analytics and machine learning use cases.\n\n    Order may be linked and joined by other tables\n\n    '\n  limitations: 'Not suitable for real-time use cases.\n\n    Data may not be used to identify individual customers.\n\n    Max data processing per day: 10 TiB\n\n    '\n  billing: 5000 USD per month\n  noticePeriod: P3M\nmodels:\n  orders:\n    description: One record per order. Includes cancelled and deleted orders.\n    type: table\n    fields:\n      order_id:\n        ref: '#/definitions/order_id'\n        title: Order ID\n        type: text\n        format: uuid\n        required: true\n        primary: true\n        unique: true\n        description: An internal ID that identifies an order in the online shop.\n        pii: true\n        classification: restricted\n        tags:\n        - orders\n        example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n      order_timestamp:\n        type: timestamp\n        required: true\n        primary: false\n        unique: false\n        description: The business timestamp in UTC when the order was successfully\n          registered in the source system and the payment was successful.\n        example: '2024-09-09T08:30:00Z'\n      order_total:\n        type: long\n        required: true\n        primary: false\n        unique: false\n        description: Total amount the smallest monetary unit (e.g., cents).\n        example: '9999'\n      customer_id:\n        type: text\n        required: false\n        primary: false\n        unique: false\n        description: Unique identifier for the customer.\n        minLength: 10\n        maxLength: 20\n      customer_email_address:\n        type: text\n        format: email\n        required: true\n        primary: false\n        unique: false\n        description: The email address, as entered by the customer. The email address\n          was not verified.\n        pii: true\n        classification: sensitive\n      processed_timestamp:\n        type: timestamp\n        required: true\n        primary: false\n        unique: false\n        description: The timestamp when the record was processed by the data platform.\n        config:\n          jsonType: string\n          jsonFormat: date-time\n  line_items:\n    description: A single article that is part of an order.\n    type: table\n    fields:\n      lines_item_id:\n        type: text\n        required: true\n        primary: true\n        unique: true\n        description: Primary key of the lines_item_id table\n      order_id:\n        ref: '#/definitions/order_id'\n        title: Order ID\n        type: text\n        format: uuid\n        required: false\n        primary: false\n        unique: false\n        references: orders.order_id\n        description: An internal ID that identifies an order in the online shop.\n        pii: true\n        classification: restricted\n        tags:\n        - orders\n        example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n      sku:\n        ref: '#/definitions/sku'\n        title: Stock Keeping Unit\n        type: text\n        required: false\n        primary: false\n        unique: false\n        description: The purchased article number\n        pattern: ^[A-Za-z0-9]{8,14}$\n        tags:\n        - inventory\n        links:\n          wikipedia: https://en.wikipedia.org/wiki/Stock_keeping_unit\n        example: '96385074'\ndefinitions:\n  order_id:\n    domain: checkout\n    name: order_id\n    title: Order ID\n    description: An internal ID that identifies an order in the online shop.\n    type: text\n    format: uuid\n    pii: true\n    classification: restricted\n    tags:\n    - orders\n    example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n  sku:\n    domain: inventory\n    name: sku\n    title: Stock Keeping Unit\n    description: \"A Stock Keeping Unit (SKU) is an internal unique identifier for\\\n      \\ an article. \\nIt is typically associated with an article's barcode, such as\\\n      \\ the EAN/GTIN.\\n\"\n    type: text\n    pattern: ^[A-Za-z0-9]{8,14}$\n    tags:\n    - inventory\n    links:\n      wikipedia: https://en.wikipedia.org/wiki/Stock_keeping_unit\n    example: '96385074'\nexamples:\n- type: csv\n  description: An example list of order records.\n  model: orders\n  data: 'order_id,order_timestamp,order_total,customer_id,customer_email_address,processed_timestamp\n\n    \"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\",\"2030-09-09T08:31:00Z\"\n\n    \"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\",\"2030-09-09T08:31:00Z\"\n\n    '\n- type: csv\n  description: An example list of line items.\n  model: line_items\n  data: 'lines_item_id,order_id,sku\n\n    \"LI-1\",\"1001\",\"5901234123457\"\n\n    \"LI-2\",\"1001\",\"4001234567890\"\n\n    \"LI-3\",\"1002\",\"5901234123457\"\n\n    \"LI-4\",\"1002\",\"2001234567893\"\n\n    \"LI-5\",\"1003\",\"4001234567890\"\n\n    \"LI-6\",\"1003\",\"5001234567892\"\n\n    \"LI-7\",\"1004\",\"5901234123457\"\n\n    \"LI-8\",\"1005\",\"2001234567893\"\n\n    \"LI-9\",\"1005\",\"5001234567892\"\n\n    \"LI-10\",\"1005\",\"6001234567891\"\n\n    '\nquality:\n  type: SodaCL\n  specification:\n    checks for orders:\n    - row_count >= 5\n    - duplicate_count(order_id) = 0\n    checks for line_items:\n    - values in (order_id) must exist in orders (order_id)\n    - row_count >= 5\nservicelevels:\n  availability:\n    description: The server is available during support hours\n    percentage: 99.9%\n  retention:\n    description: Data is retained for one year\n    period: P1Y\n    unlimited: false\n  latency:\n    description: Data is available within 25 hours after the order was placed\n    threshold: 25h\n    sourceTimestampField: orders.order_timestamp\n    processedTimestampField: orders.processed_timestamp\n  freshness:\n    description: The age of the youngest row in a table.\n    threshold: 25h\n    timestampField: orders.order_timestamp\n  frequency:\n    description: Data is delivered once a day\n    type: batch\n    interval: daily\n    cron: 0 0 * * *\n  support:\n    description: The data is available during typical business hours at headquarters\n    time: 9am to 5pm in EST on business days\n    responseTime: 1h\n  backup:\n    description: Data is backed up once a week, every Sunday at 0:00 UTC.\n    interval: weekly\n    cron: 0 0 * * 0\n    recoveryTime: 24 hours\n    recoveryPoint: 1 week\nlinks:\n  datacontractCli: https://cli.datacontract.com\ntags:\n- checkout\n- orders\n- s3\n
\n
\n
\n \n
\n
\n
\n
\n \n\n\n \n\n
\n\n\n" }, { "path": "examples/orders-latest/datacontract.yaml", "content": "dataContractSpecification: 1.2.0\nid: orders-latest\ninfo:\n title: Orders Latest\n version: 2.0.0\n description: |\n Successful customer orders in the webshop.\n All orders since 2020-01-01.\n Orders with their line items are in their current state (no history included).\n owner: Checkout Team\n contact:\n name: John Doe (Data Product Owner)\n url: https://teams.microsoft.com/l/channel/example/checkout\nservers:\n production:\n type: s3\n environment: prod\n location: s3://datacontract-example-orders-latest/v2/{model}/*.json\n format: json\n delimiter: new_line\n description: \"One folder per model. One file per day.\"\n roles:\n - name: analyst_us\n description: Access to the data for US region\n - name: analyst_cn\n description: Access to the data for China region\nterms:\n usage: |\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: |\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n policies:\n - name: privacy-policy\n url: https://example.com/privacy-policy\n - name: license\n description: External data is licensed under agreement 1234.\n url: https://example.com/license/1234\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table\n fields:\n order_id:\n $ref: '#/definitions/order_id'\n required: true\n unique: true\n primaryKey: true\n order_timestamp:\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n type: timestamp\n required: true\n examples:\n - \"2024-09-09T08:30:00Z\"\n tags: [\"business-timestamp\"]\n order_total:\n description: Total amount the smallest monetary unit (e.g., cents).\n type: long\n required: true\n examples:\n - 9999\n quality:\n - type: sql\n description: 95% of all order total values are expected to be between 10 and 499 EUR.\n query: |\n SELECT quantile_cont(order_total, 0.95) AS percentile_95\n FROM orders\n mustBeBetween: [1000, 49900]\n customer_id:\n description: Unique identifier for the customer.\n type: text\n minLength: 10\n maxLength: 20\n customer_email_address:\n description: The email address, as entered by the customer.\n type: text\n format: email\n required: true\n pii: true\n classification: sensitive\n quality:\n - type: text\n description: The email address is not verified and may be invalid.\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: email_address\n processed_timestamp:\n description: The timestamp when the record was processed by the data platform.\n type: timestamp\n required: true\n config:\n jsonType: string\n jsonFormat: date-time\n quality:\n - type: sql\n description: The maximum duration between two orders should be less that 3600 seconds\n query: |\n SELECT MAX(duration) AS max_duration FROM (SELECT EXTRACT(EPOCH FROM (order_timestamp - LAG(order_timestamp)\n OVER (ORDER BY order_timestamp))) AS duration FROM orders)\n mustBeLessThan: 3600\n - type: sql\n description: Row Count\n query: |\n SELECT count(*) as row_count\n FROM orders\n mustBeGreaterThan: 5\n examples:\n - |\n order_id,order_timestamp,order_total,customer_id,customer_email_address,processed_timestamp\n \"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\",\"2030-09-09T08:31:00Z\"\n \"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\",\"2030-09-09T08:31:00Z\"\n \"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\",\"2030-09-09T08:31:00Z\"\n \"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\",\"2030-09-09T08:31:00Z\"\n \"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\",\"2030-09-09T08:31:00Z\"\n \"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\",\"2030-09-09T08:31:00Z\"\n \"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\",\"2030-09-09T08:31:00Z\"\n \"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\",\"2030-09-09T08:31:00Z\"\n line_items:\n description: A single article that is part of an order.\n type: table\n fields:\n line_item_id:\n type: text\n description: Primary key of the lines_item_id table\n required: true\n order_id:\n $ref: '#/definitions/order_id'\n references: orders.order_id\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\n primaryKey: [\"order_id\", \"line_item_id\"]\n examples:\n - |\n line_item_id,order_id,sku\n \"LI-1\",\"1001\",\"5901234123457\"\n \"LI-2\",\"1001\",\"4001234567890\"\n \"LI-3\",\"1002\",\"5901234123457\"\n \"LI-4\",\"1002\",\"2001234567893\"\n \"LI-5\",\"1003\",\"4001234567890\"\n \"LI-6\",\"1003\",\"5001234567892\"\n \"LI-7\",\"1004\",\"5901234123457\"\n \"LI-8\",\"1005\",\"2001234567893\"\n \"LI-9\",\"1005\",\"5001234567892\"\n \"LI-10\",\"1005\",\"6001234567891\"\ndefinitions:\n order_id:\n title: Order ID\n type: text\n format: uuid\n description: An internal ID that identifies an order in the online shop.\n examples:\n - 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n tags:\n - orders\n sku:\n title: Stock Keeping Unit\n type: text\n pattern: ^[A-Za-z0-9]{8,14}$\n examples:\n - \"96385074\"\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article.\n It is typically associated with an article's barcode, such as the EAN/GTIN.\n links:\n wikipedia: https://en.wikipedia.org/wiki/Stock_keeping_unit\n tags:\n - inventory\nservicelevels:\n availability:\n description: The server is available during support hours\n percentage: 99.9%\n retention:\n description: Data is retained for one year\n period: P1Y\n unlimited: false\n latency:\n description: Data is available within 25 hours after the order was placed\n threshold: 25h\n sourceTimestampField: orders.order_timestamp\n processedTimestampField: orders.processed_timestamp\n freshness:\n description: The age of the youngest row in a table.\n threshold: 25h\n timestampField: orders.order_timestamp\n frequency:\n description: Data is delivered once a day\n type: batch # or streaming\n interval: daily # for batch, either or cron\n cron: 0 0 * * * # for batch, either or interval\n support:\n description: The data is available during typical business hours at headquarters\n time: 9am to 5pm in EST on business days\n responseTime: 1h\n backup:\n description: Data is backed up once a week, every Sunday at 0:00 UTC.\n interval: weekly\n cron: 0 0 * * 0\n recoveryTime: 24 hours\n recoveryPoint: 1 week\ntags:\n - checkout\n - orders\n - s3\nlinks:\n datacontractCli: https://cli.datacontract.com\n" }, { "path": "examples/orders-latest-nested/datacontract.html", "content": "\n\n\n Data Contract\n \n \n \n \n\n\n\n
\n\n \n\n
\n\n
\n
\n
\n
\n

\n Data Contract

\n
\n urn:orders-latest-nested\n
\n
\n \n
\n
\n
\n \n \n \n \n Show YAML\n \n
\n
\n
\n\n
\n
\n
\n
\n

Info

\n

Information about the data contract

\n
\n
\n\n
\n\n \n\n
\n
\n
Title
\n
Orders Latest (Nested)
\n
\n\n
\n
Version
\n
1.0.0
\n
\n\n \n\n \n
\n
Description
\n
\n Successful customer orders in the webshop. \nAll orders since 2020-01-01. \nOrders with their line items are in their current state (no history included).\n\n
\n
\n \n\n \n
\n
Owner
\n
\n Checkout Team\n
\n
\n \n\n \n\n \n
\n
Contact
\n
\n \n John Doe (Data Product Owner)\n \n \n \n \n \n
\n
\n \n\n
\n
\n
\n
\n\n\n \n\n\n \n
\n
\n

Terms

\n

Terms and conditions of the data contract

\n
\n
\n\n
\n\n
\n
\n
Usage
\n
\n Data can be used for reports, analytics and machine learning use cases.\nOrder may be linked and joined by other tables\n\n
\n
\n\n
\n
Limitations
\n
\n Not suitable for real-time use cases.\nData may not be used to identify individual customers.\nMax data processing per day: 10 TiB\n\n
\n
\n\n \n
\n
Billing
\n
\n 5000 USD per month\n
\n
\n \n\n \n
\n
Notice Period
\n
\n P3M\n
\n
\n \n\n \n\n
\n
\n
\n
\n \n\n\n
\n
\n
\n

\n Data Model\n

\n

The logical data model

\n
\n
\n\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n\n \n\n \n \n \n\n\n\n\n \n\n \n \n \n\n\n\n\n \n\n \n \n \n\n\n\n\n\n\n\n\n \n \n\n \n \n \n\n\n\n \n \n
\n orders\n table\n
One record per order. Includes cancelled and deleted orders.
\n
\n\n
\n \n Order ID
\n \n order_id \n \n \n
\n 		\n \n text\n \n \n \n
An internal ID that identifies an order in the online shop.
\n \n\n \n
\n Example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n
\n \n\n
\n \n primary\n \n \n required\n \n \n unique\n \n \n format:uuid\n \n \n \n \n \n \n \n \n \n \n \n restricted\n \n \n PII\n \n \n \n
\n
\n\n
\n \n order_timestamp\n
\n 		\n \n timestamp\n \n \n \n
The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n order_total\n
\n 		\n \n long\n \n \n \n
Total amount the smallest monetary unit (e.g., cents).
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n customer_id\n
\n 		\n \n text\n \n \n \n
Unique identifier for the customer.
\n \n\n \n\n
\n \n \n \n \n \n minLength:10\n \n \n maxLength:20\n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n customer_email_address\n
\n 		\n \n text\n \n \n \n
The email address, as entered by the customer. The email address was not verified.
\n \n\n \n\n
\n \n \n required\n \n \n \n format:email\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n address\n
\n 		\n \n object\n \n \n \n
The delivery address of the customer.
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n \n \n
 
\n \n
\n ↳\n
\n\n
\n \n street\n
\n 		\n \n text\n \n \n \n
The street name and house number.
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n \n \n
 
\n \n
\n ↳\n
\n\n
\n \n city\n
\n 		\n \n text\n \n \n \n
The city name.
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n \n \n
 
\n \n
\n ↳\n
\n\n
\n \n additional_lines\n
\n 		\n \n array\n \n \n \n
Additional address lines, such as floor, apartment, or company name.
\n \n\n \n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n processed_timestamp\n
\n 		\n \n timestamp\n \n \n \n
The timestamp when the record was processed by the data platform.
\n \n\n \n\n
\n \n \n required\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n
\n
\n
\n \n\n
\n
\n
\n
\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n\n \n \n \n\n\n\n \n \n
\n line_items\n table\n
A single article that is part of an order.
\n
\n\n
\n \n lines_item_id\n
\n 		\n \n text\n \n \n \n
Primary key of the lines_item_id table
\n \n\n \n\n
\n \n primary\n \n \n required\n \n \n unique\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n\n
\n \n Order ID
\n \n order_id \n \n \n
\n 		\n \n text\n \n \n \n
An internal ID that identifies an order in the online shop.
\n \n\n \n
\n Example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n
\n \n\n
\n \n \n \n \n format:uuid\n \n \n \n \n \n \n \n \n \n \n \n restricted\n \n \n PII\n \n \n \n
\n
\n\n
\n \n Stock Keeping Unit
\n \n sku \n \n \n
\n 		\n \n text\n \n \n \n
The purchased article number
\n \n\n \n
\n Example: 96385074\n
\n \n\n
\n \n \n \n \n \n \n \n pattern:^[A-Za-z0-9]{8,14}$\n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n
\n
\n
\n \n
\n\n \n
\n
\n

Definitions

\n

Domain specific definitions in the data contract

\n
\n\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n\n \n \n
\n order_id\n checkout\n
An internal ID that identifies an order in the online shop.
\n
\n
\n \n
Order ID
\n \n
order_id
\n
\n 		\n
\n text\n \n uuid\n \n
\n 		\n \n
\n Example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n
\n \n \n \n
\n \n \n \n \n \n \n \n \n \n \n restricted\n \n \n PII\n \n \n \n
\n
\n
\n
\n
\n
\n \n
\n
\n
\n
\n\n \n \n \n \n \n \n \n \n \n \n \n\n \n \n
\n sku\n inventory\n
A Stock Keeping Unit (SKU) is an internal unique identifier for an article. \nIt is typically associated with an article's barcode, such as the EAN/GTIN.\n
\n
\n
\n \n
Stock Keeping Unit
\n \n
sku
\n
\n 		\n
\n text\n \n
\n 		\n \n
\n Example: 96385074\n
\n \n \n \n
\n \n \n \n pattern:^[A-Za-z0-9]{8,14}$\n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n
\n
\n
\n \n
\n \n\n \n\n \n\n \n\n
\n
\n\n
\n Created at 27 Jun 2024 14:50:06 UTC with Data Contract CLI v0.10.8\n
\n\n
\n
\n\n \n
\n\n
\n
\n
\n
\n \n
\n
\n
\n\n

Data Contract YAML

\n
\n 
dataContractSpecification: 0.9.3\nid: urn:orders-latest-nested\ninfo:\n  title: Orders Latest (Nested)\n  version: 1.0.0\n  description: \"Successful customer orders in the webshop. \\nAll orders since 2020-01-01.\\\n    \\ \\nOrders with their line items are in their current state (no history included).\\n\"\n  owner: Checkout Team\n  contact:\n    name: John Doe (Data Product Owner)\n    url: https://teams.microsoft.com/l/channel/example/checkout\nterms:\n  usage: 'Data can be used for reports, analytics and machine learning use cases.\n\n    Order may be linked and joined by other tables\n\n    '\n  limitations: 'Not suitable for real-time use cases.\n\n    Data may not be used to identify individual customers.\n\n    Max data processing per day: 10 TiB\n\n    '\n  billing: 5000 USD per month\n  noticePeriod: P3M\nmodels:\n  orders:\n    description: One record per order. Includes cancelled and deleted orders.\n    type: table\n    fields:\n      order_id:\n        ref: '#/definitions/order_id'\n        title: Order ID\n        type: text\n        format: uuid\n        required: true\n        primary: true\n        unique: true\n        description: An internal ID that identifies an order in the online shop.\n        pii: true\n        classification: restricted\n        example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n      order_timestamp:\n        type: timestamp\n        required: true\n        primary: false\n        unique: false\n        description: The business timestamp in UTC when the order was successfully\n          registered in the source system and the payment was successful.\n      order_total:\n        type: long\n        required: true\n        primary: false\n        unique: false\n        description: Total amount the smallest monetary unit (e.g., cents).\n      customer_id:\n        type: text\n        required: false\n        primary: false\n        unique: false\n        description: Unique identifier for the customer.\n        minLength: 10\n        maxLength: 20\n      customer_email_address:\n        type: text\n        format: email\n        required: true\n        primary: false\n        unique: false\n        description: The email address, as entered by the customer. The email address\n          was not verified.\n      address:\n        type: object\n        required: false\n        primary: false\n        unique: false\n        description: The delivery address of the customer.\n        fields:\n          street:\n            type: text\n            required: false\n            primary: false\n            unique: false\n            description: The street name and house number.\n          city:\n            type: text\n            required: false\n            primary: false\n            unique: false\n            description: The city name.\n          additional_lines:\n            type: array\n            required: false\n            primary: false\n            unique: false\n            description: Additional address lines, such as floor, apartment, or company\n              name.\n            items:\n              type: text\n              required: false\n              primary: false\n              unique: false\n              description: Additional line\n      processed_timestamp:\n        type: timestamp\n        required: true\n        primary: false\n        unique: false\n        description: The timestamp when the record was processed by the data platform.\n  line_items:\n    description: A single article that is part of an order.\n    type: table\n    fields:\n      lines_item_id:\n        type: text\n        required: true\n        primary: true\n        unique: true\n        description: Primary key of the lines_item_id table\n      order_id:\n        ref: '#/definitions/order_id'\n        title: Order ID\n        type: text\n        format: uuid\n        required: false\n        primary: false\n        unique: false\n        references: orders.order_id\n        description: An internal ID that identifies an order in the online shop.\n        pii: true\n        classification: restricted\n        example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n      sku:\n        ref: '#/definitions/sku'\n        title: Stock Keeping Unit\n        type: text\n        required: false\n        primary: false\n        unique: false\n        description: The purchased article number\n        pattern: ^[A-Za-z0-9]{8,14}$\n        example: '96385074'\ndefinitions:\n  order_id:\n    domain: checkout\n    name: order_id\n    title: Order ID\n    description: An internal ID that identifies an order in the online shop.\n    type: text\n    format: uuid\n    pii: true\n    classification: restricted\n    example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n  sku:\n    domain: inventory\n    name: sku\n    title: Stock Keeping Unit\n    description: \"A Stock Keeping Unit (SKU) is an internal unique identifier for\\\n      \\ an article. \\nIt is typically associated with an article's barcode, such as\\\n      \\ the EAN/GTIN.\\n\"\n    type: text\n    pattern: ^[A-Za-z0-9]{8,14}$\n    example: '96385074'\n
\n
\n
\n \n
\n
\n
\n
\n \n\n\n \n\n
\n\n\n" }, { "path": "examples/orders-latest-nested/datacontract.yaml", "content": "dataContractSpecification: 0.9.3\nid: urn:orders-latest-nested\ninfo:\n title: Orders Latest (Nested)\n version: 1.0.0\n description: |\n Successful customer orders in the webshop. \n All orders since 2020-01-01. \n Orders with their line items are in their current state (no history included).\n owner: Checkout Team\n contact:\n name: John Doe (Data Product Owner)\n url: https://teams.microsoft.com/l/channel/example/checkout\nterms:\n usage: |\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: |\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table\n fields:\n order_id:\n $ref: '#/definitions/order_id'\n required: true\n unique: true\n primary: true\n order_timestamp:\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n type: timestamp\n required: true\n order_total:\n description: Total amount the smallest monetary unit (e.g., cents).\n type: long\n required: true\n customer_id:\n description: Unique identifier for the customer.\n type: text\n minLength: 10\n maxLength: 20\n customer_email_address:\n description: The email address, as entered by the customer. The email address was not verified.\n type: text\n format: email\n required: true\n address:\n type: object\n description: The delivery address of the customer.\n fields:\n street:\n description: The street name and house number.\n type: text\n city:\n description: The city name.\n type: text\n additional_lines:\n description: Additional address lines, such as floor, apartment, or company name.\n type: array\n items:\n type: text\n description: Additional line\n processed_timestamp:\n description: The timestamp when the record was processed by the data platform.\n type: timestamp\n required: true\n line_items:\n description: A single article that is part of an order.\n type: table\n fields:\n lines_item_id:\n type: text\n description: Primary key of the lines_item_id table\n required: true\n unique: true\n primary: true\n order_id:\n $ref: '#/definitions/order_id'\n references: orders.order_id\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\ndefinitions:\n order_id:\n domain: checkout\n name: order_id\n title: Order ID\n type: text\n format: uuid\n description: An internal ID that identifies an order in the online shop.\n example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n sku:\n domain: inventory\n name: sku\n title: Stock Keeping Unit\n type: text\n pattern: ^[A-Za-z0-9]{8,14}$\n example: \"96385074\"\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article. \n It is typically associated with an article's barcode, such as the EAN/GTIN.\n" }, { "path": "examples/time-example/datacontract.html", "content": "\n\n\n \n \n Time Data Type Example - Data Contract Specification v1.2.1\n \n\n\n
\n
Data Contract Specification v1.2.1
\n

Time Data Type Example

\n \n
\n

Overview

\n

This example demonstrates the usage of the new time data type introduced in Data Contract Specification v1.2.1. The time data type is specifically designed for storing time values without date information, making it perfect for business hours, schedules, and time-based data.

\n
\n\n
\n Note: The time data type may not be supported by all server types. Please check your specific data platform's documentation for compatibility.\n
\n\n

Data Contract Details

\n
    \n
  • ID: time-demo
    • \n
  • Title: Time Data Type Example
    • \n
  • Version: 1.0.0
    • \n
  • Owner: Data Contract Team
    • \n
\n\n
\n

Model: business_hours

\n

This model demonstrates how to use the time data type for business hours and schedules.

\n\n
\n
location_id
\n
type: string
\n
Unique identifier for the business location
\n
\n Examples:\n
\"loc_001\"
\n
\"loc_002\"
\n
\n
\n\n
\n
opening_time
\n
type: time
\n
Daily opening time for the business
\n
\n Examples:\n
\"09:00:00\"
\n
\"08:30:00\"
\n
\n
\n\n
\n
closing_time
\n
type: time
\n
Daily closing time for the business
\n
\n Examples:\n
\"17:00:00\"
\n
\"21:00:00\"
\n
\n
\n\n
\n
lunch_start
\n
type: time
\n
Start time of lunch break
\n
\n Examples:\n
\"12:00:00\"
\n
\"12:30:00\"
\n
\n
\n\n
\n
lunch_end
\n
type: time
\n
End time of lunch break
\n
\n Examples:\n
\"13:00:00\"
\n
\"13:30:00\"
\n
\n
\n
\n\n
\n

Model: shift_schedules

\n

This model demonstrates time data type usage for employee shift schedules.

\n\n
\n
shift_start
\n
type: time
\n
Start time of the employee's shift
\n
\n Examples:\n
\"08:00:00\"
\n
\"16:00:00\"
\n
\n
\n\n
\n
shift_end
\n
type: time
\n
End time of the employee's shift
\n
\n Examples:\n
\"16:00:00\"
\n
\"00:00:00\"
\n
\n
\n
\n\n

Sample Data

\n
\n Business Hours CSV Example:\n
location_id,location_name,opening_time,closing_time,lunch_start,lunch_end
\n
\"loc_001\",\"Downtown Store\",\"09:00:00\",\"17:00:00\",\"12:00:00\",\"13:00:00\"
\n
\"loc_002\",\"Mall Branch\",\"08:30:00\",\"21:00:00\",\"12:30:00\",\"13:30:00\"
\n
\"loc_003\",\"24/7 Store\",\"00:00:00\",\"23:59:59\",,
\n
\n\n
\n Shift Schedules CSV Example:\n
employee_id,shift_start,shift_end,break_start,break_end
\n
\"emp_001\",\"08:00:00\",\"16:00:00\",\"12:00:00\",\"13:00:00\"
\n
\"emp_002\",\"16:00:00\",\"00:00:00\",\"20:00:00\",\"21:00:00\"
\n
\"emp_003\",\"00:00:00\",\"08:00:00\",,
\n
\n\n

Time Data Type Characteristics

\n
    \n
  • Format: Typically follows ISO 8601 time format (HH:MM:SS)
    • \n
  • No Date Information: Contains only time components, no date
    • \n
  • 24-hour Format: Uses 24-hour clock format
    • \n
  • Optional Seconds: Can include or exclude seconds based on precision needs
    • \n
\n\n

Use Cases

\n
\n

Business Operations

\n
    \n
  • Store opening and closing hours
    • \n
  • Service availability times
    • \n
  • Break and lunch schedules
    • \n
  • Operating hours for different days
    • \n
\n
\n\n
\n

Employee Management

\n
    \n
  • Shift start and end times
    • \n
  • Break periods
    • \n
  • Work schedule definitions
    • \n
  • Time tracking data
    • \n
\n
\n\n
\n

Event Scheduling

\n
    \n
  • Meeting start and end times
    • \n
  • Event schedules
    • \n
  • Appointment times
    • \n
  • Class schedules
    • \n
\n
\n\n
\n

Transportation

\n
    \n
  • Departure and arrival times
    • \n
  • Bus and train schedules
    • \n
  • Flight times
    • \n
  • Delivery time windows
    • \n
\n
\n\n

Comparison with Other Time-Related Types

\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Data TypeDescriptionExample
timeTime only, no date\"09:00:00\"
dateDate only, no time\"2024-01-15\"
timestampDate and time with timezone\"2024-01-15T09:00:00Z\"
\n\n
\n

Testing

\n

You can test this data contract using the Data Contract CLI:

\n
datacontract test examples/time-example/datacontract.yaml
\n
\n\n
\n Important: When using the time data type, ensure your data processing tools and pipelines are compatible with time-only data. Some data platforms may require specific configurations or have limitations when working with time data types.\n
\n
\n\n " }, { "path": "examples/time-example/datacontract.yaml", "content": "dataContractSpecification: 1.2.1\nid: time-demo\ninfo:\n title: Time Data Type Example\n version: 1.0.0\n description: |\n Example demonstrating the usage of the time data type\n introduced in Data Contract Specification v1.2.1.\n owner: Data Contract Team\n contact:\n name: Data Contract Team\n url: https://github.com/datacontract/datacontract-specification\nservers:\n production:\n type: s3\n environment: prod\n location: s3://example-time-demo/{model}/*.json\n format: json\n delimiter: new_line\n description: \"Example data with time fields\"\nterms:\n usage: |\n This is an example demonstrating the new time data type.\n Data can be used for testing and educational purposes.\n limitations: |\n This is example data only and should not be used in production.\nmodels:\n business_hours:\n description: Business hours for different locations\n type: table\n fields:\n location_id:\n type: string\n description: Unique identifier for the business location\n required: true\n primaryKey: true\n examples:\n - \"loc_001\"\n - \"loc_002\"\n location_name:\n type: string\n description: Name of the business location\n required: true\n examples:\n - \"Downtown Store\"\n - \"Mall Branch\"\n opening_time:\n type: time\n description: Daily opening time for the business\n required: true\n examples:\n - \"09:00:00\"\n - \"08:30:00\"\n closing_time:\n type: time\n description: Daily closing time for the business\n required: true\n examples:\n - \"17:00:00\"\n - \"21:00:00\"\n lunch_start:\n type: time\n description: Start time of lunch break\n required: false\n examples:\n - \"12:00:00\"\n - \"12:30:00\"\n lunch_end:\n type: time\n description: End time of lunch break\n required: false\n examples:\n - \"13:00:00\"\n - \"13:30:00\"\n examples:\n - |\n location_id,location_name,opening_time,closing_time,lunch_start,lunch_end\n \"loc_001\",\"Downtown Store\",\"09:00:00\",\"17:00:00\",\"12:00:00\",\"13:00:00\"\n \"loc_002\",\"Mall Branch\",\"08:30:00\",\"21:00:00\",\"12:30:00\",\"13:30:00\"\n \"loc_003\",\"24/7 Store\",\"00:00:00\",\"23:59:59\",,\n shift_schedules:\n description: Employee shift schedules\n type: table\n fields:\n employee_id:\n type: string\n description: Unique identifier for the employee\n required: true\n primaryKey: true\n examples:\n - \"emp_001\"\n - \"emp_002\"\n shift_start:\n type: time\n description: Start time of the employee's shift\n required: true\n examples:\n - \"08:00:00\"\n - \"16:00:00\"\n shift_end:\n type: time\n description: End time of the employee's shift\n required: true\n examples:\n - \"16:00:00\"\n - \"00:00:00\"\n break_start:\n type: time\n description: Start time of the break period\n required: false\n examples:\n - \"12:00:00\"\n - \"20:00:00\"\n break_end:\n type: time\n description: End time of the break period\n required: false\n examples:\n - \"13:00:00\"\n - \"21:00:00\"\n examples:\n - |\n employee_id,shift_start,shift_end,break_start,break_end\n \"emp_001\",\"08:00:00\",\"16:00:00\",\"12:00:00\",\"13:00:00\"\n \"emp_002\",\"16:00:00\",\"00:00:00\",\"20:00:00\",\"21:00:00\"\n \"emp_003\",\"00:00:00\",\"08:00:00\",,\ntags:\n - example\n - time\n - v1.2.1\nlinks:\n datacontractCli: https://cli.datacontract.com " }, { "path": "examples/variant-json-example/datacontract.yaml", "content": "dataContractSpecification: 1.2.1\nid: variant-json-demo\ninfo:\n title: Variant and JSON Data Types Example\n version: 1.0.0\n description: |\n Example demonstrating the usage of variant and json data types\n introduced in Data Contract Specification v1.2.1.\n owner: Data Contract Team\n contact:\n name: Data Contract Team\n url: https://github.com/datacontract/datacontract-specification\nservers:\n production:\n type: s3\n environment: prod\n location: s3://example-variant-json-demo/{model}/*.json\n format: json\n delimiter: new_line\n description: \"Example data with variant and json fields\"\nterms:\n usage: |\n This is an example demonstrating the new variant and json data types.\n Data can be used for testing and educational purposes.\n limitations: |\n This is example data only and should not be used in production.\nmodels:\n user_profiles:\n description: User profiles with variant and JSON data fields\n type: table\n fields:\n user_id:\n type: string\n description: Unique identifier for the user\n required: true\n primaryKey: true\n examples:\n - \"user_123\"\n - \"user_456\"\n profile_data:\n type: variant\n description: Semi-structured profile data that can contain various types of information\n required: false\n examples:\n - \"John Doe\"\n - 25\n - true\n - {\"preferences\": {\"theme\": \"dark\", \"language\": \"en\"}}\n metadata:\n type: json\n description: JSON-formatted metadata about the user profile\n required: false\n examples:\n - '{\"created_at\": \"2024-01-15T10:30:00Z\", \"source\": \"web_form\", \"version\": 1}'\n - '{\"tags\": [\"premium\", \"verified\"], \"settings\": {\"notifications\": true}}'\n preferences:\n type: json\n description: User preferences stored as JSON\n required: false\n examples:\n - '{\"theme\": \"dark\", \"language\": \"en\", \"timezone\": \"UTC\"}'\n - '{\"notifications\": {\"email\": true, \"sms\": false, \"push\": true}}'\n examples:\n - |\n user_id,profile_data,metadata,preferences\n \"user_123\",\"John Doe\",'{\"created_at\": \"2024-01-15T10:30:00Z\", \"source\": \"web_form\"}','{\"theme\": \"dark\", \"language\": \"en\"}'\n \"user_456\",25,'{\"tags\": [\"premium\"], \"version\": 2}','{\"notifications\": {\"email\": true, \"sms\": false}}'\n \"user_789\",true,'{\"source\": \"api\", \"verified\": true}','{\"theme\": \"light\", \"timezone\": \"EST\"}'\ntags:\n - example\n - variant\n - json\n - v1.2.1\nlinks:\n datacontractCli: https://cli.datacontract.com " }, { "path": "gen-openapi-yaml", "content": "#!/bin/bash\n\n# INSTALL BEFORE\n# npm install -g @openapi-contrib/json-schema-to-openapi-schema\n# brew install yq\n\njson-schema-to-openapi-schema convert datacontract.schema.json > datacontract.schema.openapi-format.json\nyq --input-format=json --output-format=yaml --prettyPrint datacontract.schema.openapi-format.json > datacontract.schema.openapi-format.yaml\necho \"Compare 'datacontract.schema.openapi-format.yaml' with openapi.yaml of the Data Mesh Manager\"\necho \"Prepend 'DataContract:\\\\n' and match the indendation correctly. Then, compare in IntelliJ\" \n" }, { "path": "versions/0.9.0/README.md", "content": "# Data Contract Specification\n\n![datacontract.png](images/datacontract.png)\n\nData contracts bring data providers and data consumers together.\n\nA _data contract_ is a document that defines the structure, format, semantics, quality, and terms of use for exchanging data between a data provider and their consumers. A data contract is implemented by a data product's output port or other data technologies. Data contracts can also be used for the input port to specify the expectations of data dependencies and verify given guarantees.\n\nThe _data contract specification_ defines a YAML format to describe attributes of provided data sets. It is data platform neutral, yet supports well-known formats to express schemas (e.g., dbt models, JSON Schema, Protobuf, SQL DDL) and quality tests (e.g., SodaCL, SQL queries) to avoid unnecessary abstractions. The data contract specification is an open initiative to define a common data contract format. Think of an [OpenAPI specification](https://www.openapis.org/), but for data sets.\n\nData contracts come into play when data is exchanged between different teams or organizational units, such as in a [data mesh architecture](https://www.datamesh-architecture.com/).\nFirst, and foremost, data contracts are a communication tool to express a common understanding of how data should be structured and interpreted. They make semantic and quality expectations explicit. They are often created in [workshops](/workshop). Later in development and production, they also serve as the basis for code generation, testing, schema validations, quality checks, monitoring, access control, and computational governance policies.\n\n_Note: The term \"data contract\" refers to a specification that is usually owned by the data provider and thus does not align with a \"contract\" in a legal sense as a mutual agreement between two parties. The term \"contract\" may be somewhat misleading, but it is how it is used in practice. The mutual agreement between one data provider and one data consumer is the \"data usage agreement\" that refers to a data contract. Data usage agreements have a defined lifecycle, start/end date, and help the data provider to track who accesses their data and for which purposes._\n\nThe specification is inspired by [AIDA User Group's Open Data Contract Standard](https://github.com/AIDAUserGroup/open-data-contract-standard), (formerly [PayPal's Data Contract Template](https://github.com/paypal/data-contract-template/blob/main/docs/README.md)) and Data Mesh Manager's [Data Contract API](https://www.datamesh-manager.com).\nIt follows [OpenAPI](https://www.openapis.org/) and [AsyncAPI](https://www.asyncapi.com/) conventions.\n\nVersion\n---\n\n0.9.0\n\nExample\n---\n\n[![Open in Data Contract Studio](https://img.shields.io/badge/open%20in-Data%20Contract%20Studio-blue)](https://studio.datacontract.com/)\n\n```yaml\ndataContractSpecification: 0.9.0\nid: urn:datacontract:checkout:orders-latest-npii\ninfo:\n title: Orders Latest NPII\n version: 1.0.0\n description: Successful customer orders in the webshop. All orders since 2020-01-01. Orders with their line items are in their current state (no history included). PII data is removed.\n owner: Checkout Team\n contact:\n name: John Doe (Data Product Owner)\n email: john.doe@example.com\nservers:\n production:\n type: BigQuery\n project: acme_orders_prod\n dataset: bigquery_orders_latest_npii_v1\nterms:\n usage: >\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: >\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n billing: 5000 USD per month\n noticePeriod: P3M\nschema:\n type: dbt # the specification format: dbt, bigquery, avro, protobuf, sql, json-schema, custom\n specification: # expressed as string or inline yaml or via \"$ref: model.yaml\"\n version: 2 \n description: The subset of the output port's data model that we agree to use\n models:\n - name: orders\n description: >\n One record per order. Includes cancelled and deleted orders.\n columns:\n - name: order_id\n data_type: string\n description: Primary key of the orders table\n - name: order_timestamp\n data_type: timestamptz\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n - name: order_total\n data_type: integer\n description: \"Total amount of the order in the smallest monetary unit (e.g., cents).\"\n - name: line_items\n description: >\n The items that are part of an order\n columns:\n - name: lines_item_id\n data_type: string\n description: Primary key of the lines_item_id table\n - name: order_id\n data_type: string\n description: Foreign key to the orders table\n - name: sku\n data_type: string\n description: The purchased article number\nexamples:\n - type: csv # csv, json, yaml, custom\n model: orders\n data: |- # expressed as string or inline yaml or via \"$ref: data.csv\"\n order_id,order_timestamp,order_total\n \"1001\",\"2023-09-09T08:30:00Z\",2500\n \"1002\",\"2023-09-08T15:45:00Z\",1800\n \"1003\",\"2023-09-07T12:15:00Z\",3200\n \"1004\",\"2023-09-06T19:20:00Z\",1500\n \"1005\",\"2023-09-05T10:10:00Z\",4200\n \"1006\",\"2023-09-04T14:55:00Z\",2800\n \"1007\",\"2023-09-03T21:05:00Z\",1900\n \"1008\",\"2023-09-02T17:40:00Z\",3600\n \"1009\",\"2023-09-01T09:25:00Z\",3100\n \"1010\",\"2023-08-31T22:50:00Z\",2700\n - type: csv\n model: line_items\n data: |-\n lines_item_id,order_id,sku\n \"1\",\"1001\",\"5901234123457\"\n \"2\",\"1001\",\"4001234567890\"\n \"3\",\"1002\",\"5901234123457\"\n \"4\",\"1002\",\"2001234567893\"\n \"5\",\"1003\",\"4001234567890\"\n \"6\",\"1003\",\"5001234567892\"\n \"7\",\"1004\",\"5901234123457\"\n \"8\",\"1005\",\"2001234567893\"\n \"9\",\"1005\",\"5001234567892\"\n \"10\",\"1005\",\"6001234567891\"\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - freshness(order_timestamp) < 24h\n - row_count > 500000\n - duplicate_count(order_id) = 0\n checks for line_items:\n - row_count > 500000\n```\n\nSchema\n---\n\n[JSON Schema](https://github.com/datacontract/datacontract-specification/blob/main/datacontract.schema.json) of the Data Contract Specification.\n\n### Data Contract Object\n\nThis is the root document.\n\nIt is _RECOMMENDED_ that the root document be named: `datacontract.yaml`.\n\n| Field | Type | Description |\n|---------------------------|------------------------------------|-------------------------------------------------------------------------------------------------------|\n| dataContractSpecification | `string` | REQUIRED. Specifies the Data Contract Specification being used. |\n| id | `string` | REQUIRED. An organization-wide unique technical identifier, such as a UUID, URN, slug, string, or number |\n| info | [Info Object](#info-object) | REQUIRED. Specifies the metadata of the data contract. May be used by tooling. |\n| servers | [Servers Object](#servers-object) | Specifies the servers of the data contract. |\n| terms | [Terms Object](#terms-object) | Specifies the terms and conditions of the data contract. |\n| schema | [Schema Object](#schema-object) | Specifies the data contract schema. The specification supports different schemas. |\n| examples | [Examples Object](#examples-object) | Specifies example data sets for the schema. The specification supports different example types. |\n| quality | [Quality Object](#quality-object) | Specifies the quality attributes and checks. The specification supports different quality check DSLs. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n\n### Info Object\n\nMetadata and life cycle information about the data contract.\n\n\n| Field | Type | Description |\n|---------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| title | `string` | REQUIRED. The title of the data contract. |\n| version | `string` | REQUIRED. The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version). |\n| description | `string` | A description of the data contract. |\n| owner | `string` | The owner or team responsible for managing the data contract and providing the data. |\n| dataProduct | `string` | The identifier of the data product that contains the output port providing the data. |\n| outputPort | `string` | DEPRECATED. The identifier of the output port that implements the data contract. |\n| contact | [Contact Object](#contact-object) | Contact information for the data contract. |\n\n\n\n\n### Contact Object\n\nContact information for the data contract.\n\n| Field | Type | Description |\n|-------|----------|-------------------------------------------------------------------------------------------------------|\n| name | `string` | The identifying name of the contact person/organization. |\n| url | `string` | The URL pointing to the contact information. This _MUST_ be in the form of a URL. |\n| email | `string` | The email address of the contact person/organization. This _MUST_ be in the form of an email address. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n### Servers Object\n\nInformation about the servers.\n\nThe Servers Object is a map of [Server Objects](#server-object).\n\n### Server Object\n\nThe fields are dependent on the defined type.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `s3`, `redshift`, `snowflake`, `databricks`, `kafka` |\n| description | `string` | An optional string describing the server. |\n\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### BigQuery Server Object\n\n| Field | Type | Description |\n|---------|----------|-------------|\n| type | `string` | `bigquery` |\n| project | `string` | |\n| dataset | `string` | |\n\n#### S3 Server Object\n\n| Field | Type | Description |\n|----------|----------|--------------------------------|\n| type | `string` | `s3` |\n| location | `string` | S3 URL, starting with `s3://` |\n\nExample:\n\n```yaml\nservers:\n production:\n type: s3\n location: s3://acme-orders-prod/orders/\n```\n\n\n#### Redshift Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `redshift` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Snowflake Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `snowflake` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Databricks Server Object\n\n| Field | Type | Description |\n|----------|----------|--------------|\n| type | `string` | `databricks` |\n| share | `string` | |\n\n#### Kafka Server Object\n\n| Field | Type | Description |\n|-------|----------|-------------|\n| type | `string` | `kafka` |\n| host | `string` | |\n| topic | `string` | |\n\n\n\n### Terms Object\n\nThe terms and conditions of the data contract.\n\n| Field | Type | Description |\n|----------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| usage | `string` | The usage describes the way the data is expected to be used. Can contain business and technical information. |\n| limitations | `string` | The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for. |\n| billing | `string` | The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use. |\n| noticePeriod | `string` | The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., `P3M` for a period of three months. |\n\n\n### Schema Object\n\nThe schema of the data contract describes the syntax and semantics of provided data sets.\nAs the type of the output port depends on the data platform, multiple schema specifications are supported.\n\nA schema may define a single table, a collection of tables as a dataset, a file structure, or any arbitrary structure.\n\nTo avoid unnecessary abstractions, the data contract specification supports existing well-known formats. Some schema types, such as `dbt`, also support defining tests and additional metadata.\n\n\n| Field | Type | Description |\n| ----- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `dbt`, `bigquery`, `json-schema`, `sql-ddl`, `avro`, `protobuf`, `custom` |\n| specification | [dbt Schema Object](#dbt-schema-object) \\|
[BigQuery Schema Object](#bigquery-schema-object) \\|
[JSON Schema Schema Object](#bigquery-schema-object) \\|
[SQL DDL Schema Object](#sql-ddl-schema-object) \\|
`string` | REQUIRED. The specification of the schema. The schema specification can be encoded as a string or as inline YAML. |\n\n\n#### dbt Schema Object\n\nhttps://docs.getdbt.com/reference/model-properties\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: dbt\n specification:\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\nExample (string):\n\n```yaml\nschema:\n type: dbt\n specification: |-\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\n#### BigQuery Schema Object\n\nThe schema structure is defined by the [Google BigQuery Table](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#resource:-table) object. You can extract such a Table object via the [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) endpoint.\n\nInstead of providing a single Table object, you can also provide an array of such objects. Be aware that [tables.list](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list) only returns a subset of the full Table object. You need to call every Table object via [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) to get the full Table object, including the actual schema.\n\nLearn more: [Google BigQuery REST Reference v2](https://cloud.google.com/bigquery/docs/reference/rest)\n\n\n\nExample:\n\n```yaml\nschema:\n type: bigquery\n specification: |-\n {\n \"tableReference\": {\n \"projectId\": \"my-project\",\n \"datasetId\": \"my_dataset\",\n \"tableId\": \"my_table\"\n },\n \"description\": \"This is a description\",\n \"type\": \"TABLE\",\n \"schema\": {\n \"fields\": [\n {\n \"name\": \"name\",\n \"type\": \"STRING\",\n \"mode\": \"NULLABLE\",\n \"description\": \"This is a description\"\n }\n ]\n }\n }\n```\n\n#### JSON Schema Schema Object\n\nJSON Schema can be defined as JSON or rendered as YAML, following the [OpenAPI Schema Object dialect](https://spec.openapis.org/oas/v3.1.0#properties)\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: json-schema\n specification:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: object\n properties:\n order_id:\n type: string\n description: Primary key of the orders table\n order_timestamp:\n type: string\n format: date-time\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total:\n type: integer\n description: Total amount of the order in the smallest monetary unit (e.g., cents).\n line_items:\n type: object\n properties:\n lines_item_id:\n type: string\n description: Primary key of the lines_item_id table\n order_id:\n type: string\n description: Foreign key to the orders table\n sku:\n type: string\n description: The purchased article number\n```\n\nExample (string):\n\n```yaml\nschema:\n type: json-schema\n specification: |-\n {\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"orders\": {\n \"type\": \"object\",\n \"description\": \"One record per order. Includes cancelled and deleted orders.\",\n \"properties\": {\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the orders table\"\n },\n \"order_timestamp\": {\n \"type\": \"string\",\n \"format\": \"date-time\",\n \"description\": \"The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\"\n },\n \"order_total\": {\n \"type\": \"integer\",\n \"description\": \"Total amount of the order in the smallest monetary unit (e.g., cents).\"\n }\n },\n \"required\": [\"order_id\", \"order_timestamp\", \"order_total\"]\n },\n \"line_items\": {\n \"type\": \"object\",\n \"properties\": {\n \"lines_item_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the lines_item_id table\"\n },\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Foreign key to the orders table\"\n },\n \"sku\": {\n \"type\": \"string\",\n \"description\": \"The purchased article number\"\n }\n },\n \"required\": [\"lines_item_id\", \"order_id\", \"sku\"]\n }\n },\n \"required\": [\"orders\", \"line_items\"]\n }\n```\n\n#### SQL DDL Schema Object\n\nClassical SQL DDLs can be used to describe the structure.\n\n\nExample (string):\n\n```yaml\nschema:\n type: sql-ddl\n specification: |-\n -- One record per order. Includes cancelled and deleted orders.\n CREATE TABLE orders (\n order_id TEXT PRIMARY KEY, -- Primary key of the orders table\n order_timestamp TIMESTAMPTZ NOT NULL, -- The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total INTEGER NOT NULL -- Total amount of the order in the smallest monetary unit (e.g., cents)\n );\n \n -- The items that are part of an order\n CREATE TABLE line_items (\n lines_item_id TEXT PRIMARY KEY, -- Primary key of the lines_item_id table\n order_id TEXT REFERENCES orders(order_id), -- Foreign key to the orders table\n sku TEXT NOT NULL -- The purchased article number\n );\n\n```\n\n### Examples Object\n\nThe Examples Object is an array of [Example Objects](#examples-object).\n\n### Example Object\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `csv`, `json`, `yaml`, `custom` |\n| description | `string` | An optional string describing the example. |\n| model | `string` | The reference to the model in the schema, e.g. a table name. | \n| data | `string` | Example data for this model. |\n\nExample:\n\n```yaml\nexamples:\n- type: csv\n model: orders\n data: |-\n order_id,order_timestamp,order_total\n \"1001\",\"2023-09-09T08:30:00Z\",2500\n \"1002\",\"2023-09-08T15:45:00Z\",1800\n \"1003\",\"2023-09-07T12:15:00Z\",3200\n \"1004\",\"2023-09-06T19:20:00Z\",1500\n \"1005\",\"2023-09-05T10:10:00Z\",4200\n \"1006\",\"2023-09-04T14:55:00Z\",2800\n \"1007\",\"2023-09-03T21:05:00Z\",1900\n \"1008\",\"2023-09-02T17:40:00Z\",3600\n \"1009\",\"2023-09-01T09:25:00Z\",3100\n \"1010\",\"2023-08-31T22:50:00Z\",2700\n```\n\n### Quality Object\n\nThe quality object contains quality attributes and checks.\n\n| Field | Type | Description |\n| ----- |-------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `SodaCL`, `montecarlo`, `custom` |\n| specification | [SodaCL Quality Object](#sodacl-quality-object) \\|
[Monte Carlo Schema Object](#monte-carlo-quality-object) \\|
`string` | REQUIRED. The specification of the quality attributes. The quality specification can be encoded as a string or as inline YAML. |\n\n\n#### SodaCL Quality Object\n\nQuality attributes in [Soda Checks Language](https://docs.soda.io/soda-cl/soda-cl-overview.html).\n\nThe `specification` represents the content of a `checks.yml` file.\n\nExample (inline):\n\n```yaml\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, dbt-tests, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - row_count > 0\n - duplicate_count(order_id) = 0\n checks for line_items:\n - row_count > 0\n```\n\nExample (string):\n\n```yaml\nquality:\n type: SodaCL\n specification: |-\n checks for search_queries:\n - freshness(search_timestamp) < 1d\n - row_count > 100000\n - missing_count(search_query) = 0\n```\n\n#### Monte Carlo Quality Object\n\nQuality attributes defined as Monte Carlos [Monitors as Code](https://docs.getmontecarlo.com/docs/monitors-as-code).\n\nThe `specification` represents the content of a `montecarlo.yml` file.\n\nExample (string):\n\n```yaml\nquality:\n type: montecarlo\n specification: |-\n montecarlo:\n field_health:\n - table: project:dataset.table_name\n timestamp_field: created\n dimension_tracking:\n - table: project:dataset.table_name\n timestamp_field: created\n field: order_status\n```\n\n### Specification Extensions\n\nWhile the Data Contract Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points.\n\nA custom fields can be added with any name. The value can be null, a primitive, an array or an object.\n\n### Design Principles\n\nThe Data Contract Specification follows these design principles:\n\n- Is an open standard and its serialization can be versioned in git\n- Follows OpenAPI and AsyncAPI conventions so that it feels immediately familiar\n- Supports tooling by being machine-readable\n- Supports existing well-known formats to avoid unnecessary abstractions\n- Supports contract-first approaches\n- Supports code-first approaches\n\nTooling\n---\n- [Data Contract Studio](https://studio.datacontract.com/) is a free web tool to develop and share data contracts.\n- [Data Contract CLI](https://github.com/datacontract/cli) is a free CLI tool to help you create, develop, and maintain your data contracts.\n- [Data Mesh Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data products and data contracts. It supports the data contract specification and allows the user to import or export data contracts using this specification.\n\n\nOther Data Contract Specifications\n---\n- [AIDA User Group's Open Data Contract Standard](https://github.com/AIDAUserGroup/open-data-contract-standard)\n- [PayPal's Data Contract Template](https://github.com/paypal/data-contract-template/blob/main/docs/README.md)\n\nLiterature\n---\n- [Driving Data Quality with Data Contracts](https://www.amazon.com/dp/B0C37FPH3D) by Andrew Jones\n\nAuthors\n---\nThe Data Contract Specification was originally created by [Jochen Christ](https://www.linkedin.com/in/jochenchrist/) and [Dr. Simon Harrer](https://www.linkedin.com/in/simonharrer/), and is currently maintained by them.\n\n\nContributing\n---\nContributions are welcome! Please open an issue or a pull request.\n\nLicense\n---\n[MIT License](LICENSE)\n\n\n\n" }, { "path": "versions/0.9.0/datacontract.init.yaml", "content": "dataContractSpecification: 0.9.0\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# my-stage:\n# type: bigquery\n# project:\n# dataset:\n\n#servers:\n# my-stage:\n# type: s3\n# location: s3://\n\n#servers:\n# my-stage:\n# type: redshift\n# account:\n# database:\n# schema:\n\n#servers:\n# my-stage:\n# type: snowflake\n# account:\n# database:\n# schema:\n\n#servers:\n# my-stage:\n# type: databricks\n# share:\n\n#servers:\n# my-stage:\n# type: kafka\n# host:\n# topic:\n\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### schema\n\n#schema:\n# type: dbt\n# specification:\n# version:\n# models:\n# - name:\n# description:\n# columns:\n# - name:\n# type:\n# description:\n# tests:\n\n#schema:\n# type: dbt\n# specification: |-\n# version:\n# models:\n# - name:\n# description:\n# columns:\n# - name:\n# type:\n# description:\n# tests:\n\n#schema:\n# type: dbt\n# specification: \"$ref: model.yaml\"\n\n#schema:\n# type: bigquery\n# specification: |-\n# {\n# \"tableReference\": {\n# \"projectId\": \"my-project\",\n# \"datasetId\": \"my_dataset\",\n# \"tableId\": \"my_table\"\n# },\n# \"description\": \"This is a description\",\n# \"type\": \"TABLE\",\n# \"schema\": {\n# \"fields\": [\n# {\n# \"name\": \"name\",\n# \"type\": \"STRING\",\n# \"mode\": \"NULLABLE\",\n# \"description\": \"This is a description\"\n# }\n# ]\n# }\n# }\n\n#schema:\n# type: json-schema\n# specification:\n# my-table:\n# description:\n# type: object\n# properties:\n# id:\n# type: string\n# description:\n\n#schema:\n# type: json-schema\n# specification: |-\n# {\n# \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n# \"type\": \"object\",\n# \"properties\": {\n# \"my_table\": {\n# \"type\": \"object\",\n# \"description\": \"\",\n# \"properties\": {\n# \"id\": {\n# \"type\": \"string\",\n# \"description\": \"\"\n# },\n# \"required\": [\"id\"]\n# }\n# },\n# \"required\": [\"my-table\"]\n# }\n\n#schema:\n# type: sql-ddl\n# specification: |-\n# CREATE TABLE my_table (\n# id TEXT PRIMARY KEY\n# );\n\n#schema:\n# type: avro\n# specification:\n# User:\n# type: record\n# name: MyTable\n# fields:\n# - name: id\n# type: string\n\n#schema:\n# type: avro\n# specification: |-\n# {\n# \"type\": \"record\",\n# \"name\": \"MyTable\",\n# \"fields\": [\n# {\n# \"name\": \"name\",\n# \"type\": \"string\"\n# }\n# ]\n# }\n\n#schema:\n# type: protobuf\n# specification: |-\n# message MyTable {\n# string id = 1;\n# }\n\n#schema:\n# type: custom\n# specification:\n\n\n### examples\n\n#examples:\n# - type: csv\n# model: my_table\n# data: |-\n# id,timestamp,amount\n# \"1001\",\"2023-09-09T08:30:00Z\",2500\n# \"1002\",\"2023-09-08T15:45:00Z\",1800\n#\n#examples:\n# - type: csv\n# model: my_table\n# data: \"$ref: data.csv\"\n\n#examples:\n# - type: json\n# model: my_table\n# data: |-\n# [\n# {\n# \"id\": \"1001\",\n# \"timestamp\": \"2023-09-09T08:30:00Z\",\n# \"amount\": 2500\n# },\n# {\n# \"id\": \"1002\",\n# \"timestamp\": \"2023-09-08T15:45:00Z\",\n# \"amount\": 1800\n# }\n# ]\n\n#examples:\n# - type: yaml\n# model: my_table\n# data:\n# - id: 1001\n# timestamp: 2023-09-09T08:30:00Z\n# amount: 2500\n# - id: 1002\n# timestamp: 2023-09-08T15:45:00Z\n# amount: 1800\n\n#examples:\n# - type: custom\n# model: my_table\n# data: |-\n\n\n### quality\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_table:\n# - duplicate_count(order_id) = 0\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_table: |-\n# - duplicate_count(id) = 0\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_table: \"$ref: checks.yaml\"\n\n#quality:\n# type: montecarlo\n# specification: |-\n# montecarlo:\n# field_health:\n# - table: my_project:my_dataset.my_table\n# fields:\n# - id\n# - timestamp\n# - amount\n# timestamp_field: timestamp\n\n#quality:\n# type: custom\n# specification: |-\n" }, { "path": "versions/0.9.0/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"enum\": [\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"dataProduct\": {\n \"type\": \"string\",\n \"description\": \"The data product that contains the output port providing the data.\"\n },\n \"outputPort\": {\n \"type\": \"string\",\n \"description\": \"The output port that implements the data contract.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\"\n }\n },\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"bigquery\",\n \"BigQuery\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"project\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"project\",\n \"dataset\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"s3\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"An optional string describing the server. Must be in the form of a URL.\"\n }\n },\n \"required\": [\n \"type\",\n \"location\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"redshift\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"snowflake\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"databricks\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"share\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"share\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"kafka\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"host\",\n \"topic\"\n ]\n }\n ]\n },\n \"description\": \"Information about the servers.\"\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n }\n },\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"dbt\",\n \"bigquery\",\n \"json-schema\",\n \"sql-ddl\",\n \"avro\",\n \"protobuf\",\n \"custom\"\n ],\n \"description\": \"The type of the schema. Typical values are: dbt, bigquery, json-schema, sql-ddl, avro, protobuf, custom.\"\n },\n \"specification\": {\n \"anyOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the schema as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the schema as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The schema of the data contract describes the syntax and semantics of provided data sets. It supports different schema types.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"csv\",\n \"json\",\n \"yaml\",\n \"custom\"\n ],\n \"description\": \"The type of the example data. Well-known types are: csv, json, yaml, custom.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the example.\"\n },\n \"model\": {\n \"type\": \"string\",\n \"description\": \"The reference to the model in the schema, e.g., a table name.\"\n },\n \"data\": {\n \"type\": \"string\",\n \"description\": \"Example data for this model.\"\n }\n },\n \"required\": [\n \"type\",\n \"model\",\n \"data\"\n ]\n },\n \"description\": \"The Examples Object is an array of Example Objects.\"\n },\n \"quality\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"SodaCL\",\n \"montecarlo\",\n \"custom\"\n ],\n \"description\": \"The type of the quality check. Typical values are: SodaCL, montecarlo, custom.\"\n },\n \"specification\": {\n \"anyOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the quality attributes as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the quality attributes as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The quality object contains quality attributes and checks.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ]\n}\n" }, { "path": "versions/0.9.1/README.md", "content": "# Data Contract Specification\n\n![datacontract.png](images/datacontract.png)\n\nData contracts bring data providers and data consumers together.\n\nA _data contract_ is a document that defines the structure, format, semantics, quality, and terms of use for exchanging data between a data provider and their consumers. \nA data contract is implemented by a data product's output port or other data technologies. \nData contracts can also be used for the input port to specify the expectations of data dependencies and verify given guarantees.\n\nThe _data contract specification_ defines a YAML format to describe attributes of provided data sets. \nIt is data platform neutral and can be used with any data platform, such as AWS S3, Google BigQuery, Microsoft Fabric, Databricks, and Snowflake. \nThe data contract specification is an open initiative to define a common data contract format. \nIt follows [OpenAPI](https://www.openapis.org/) and [AsyncAPI](https://www.asyncapi.com/) conventions.\n\nData contracts come into play when data is exchanged between different teams or organizational units, such as in a [data mesh architecture](https://www.datamesh-architecture.com/). \nFirst, and foremost, data contracts are a communication tool to express a common understanding of how data should be structured and interpreted. \nThey make semantic and quality expectations explicit. \nThey are often created collaboratively in [workshops](/workshop) together with data providers and data consumers. \nLater in development and production, they also serve as the basis for code generation, testing, schema validations, quality checks, monitoring, access control, and computational governance policies.\n\nThe specification comes along with the [Data Contract CLI](https://github.com/datacontract/cli), an open-source tool to develop, validate, and enforce data contracts.\n\n_Note: The term \"data contract\" refers to a specification that is usually owned by the data provider and thus does not align with a \"contract\" in a legal sense as a mutual agreement between two parties. \nThe term \"contract\" may be somewhat misleading, but it is how it is used in practice. \nThe mutual agreement between one data provider and one data consumer is the \"data usage agreement\" that refers to a data contract. \nData usage agreements have a defined lifecycle, start/end date, and help the data provider to track who accesses their data and for which purposes._\n\nVersion\n---\n\n0.9.1 ([Changelog](CHANGELOG.md))\n\nExample\n---\n\n[![Open in Data Contract Studio](https://img.shields.io/badge/open%20in-Data%20Contract%20Studio-blue)](https://studio.datacontract.com/)\n\n```yaml\ndataContractSpecification: 0.9.1\nid: urn:datacontract:checkout:orders-latest-npii\ninfo:\n title: Orders Latest NPII\n version: 1.0.0\n description: Successful customer orders in the webshop. All orders since 2020-01-01. Orders with their line items are in their current state (no history included). PII data is removed.\n owner: Checkout Team\n contact:\n name: John Doe (Data Product Owner)\n email: john.doe@example.com\nservers:\n production:\n type: BigQuery\n project: acme_orders_prod\n dataset: bigquery_orders_latest_npii_v1\nterms:\n usage: >\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: >\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table \n fields:\n order_id:\n $ref: '#/definitions/order_id'\n order_timestamp:\n type: timestamp\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total:\n type: long\n description: Total amount the smallest monetary unit (e.g., cents).\n line_items:\n description: A single article that is part of an order. \n type: table\n fields:\n lines_item_id:\n type: string\n description: Primary key of the lines_item_id table\n order_id:\n $ref: '#/definitions/order_id'\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\ndefinitions:\n order_id:\n domain: checkout\n name: order_id\n title: Order ID\n type: string\n description: An internal ID that identifies an order in the online shop.\n example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n sku:\n domain: inventory\n name: sku\n title: Stock Keeping Unit\n type: string\n example: AC1212ME1\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article. \n It is typically associated with an article's barcode, such as the EAN/GTIN.\nexamples:\n - type: csv # csv, json, yaml, custom\n model: orders\n data: |- # expressed as string or inline yaml or via \"$ref: data.csv\"\n order_id,order_timestamp,order_total\n \"1001\",\"2023-09-09T08:30:00Z\",2500\n \"1002\",\"2023-09-08T15:45:00Z\",1800\n \"1003\",\"2023-09-07T12:15:00Z\",3200\n \"1004\",\"2023-09-06T19:20:00Z\",1500\n \"1005\",\"2023-09-05T10:10:00Z\",4200\n \"1006\",\"2023-09-04T14:55:00Z\",2800\n \"1007\",\"2023-09-03T21:05:00Z\",1900\n \"1008\",\"2023-09-02T17:40:00Z\",3600\n \"1009\",\"2023-09-01T09:25:00Z\",3100\n \"1010\",\"2023-08-31T22:50:00Z\",2700\n - type: csv\n model: line_items\n data: |-\n lines_item_id,order_id,sku\n \"1\",\"1001\",\"5901234123457\"\n \"2\",\"1001\",\"4001234567890\"\n \"3\",\"1002\",\"5901234123457\"\n \"4\",\"1002\",\"2001234567893\"\n \"5\",\"1003\",\"4001234567890\"\n \"6\",\"1003\",\"5001234567892\"\n \"7\",\"1004\",\"5901234123457\"\n \"8\",\"1005\",\"2001234567893\"\n \"9\",\"1005\",\"5001234567892\"\n \"10\",\"1005\",\"6001234567891\"\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - freshness(order_timestamp) < 24h\n - row_count > 500000\n - duplicate_count(order_id) = 0\n checks for line_items:\n - row_count > 500000\n```\n\nSchema\n---\n\n- [Data Contract Object](#data-contract-object)\n- [Info Object](#info-object)\n- [Contact Object](#contact-object)\n- [Server Object](#server-object)\n- [Terms Object](#terms-object)\n- [Model Object](#model-object)\n- [Field Object](#field-object)\n- [Definition Object](#definition-object)\n- [Schema Object](#schema-object)\n- [Example Object](#example-object)\n- [Quality Object](#quality-object)\n- [Data Types](#data-types)\n- [Specification Extensions](#specification-extensions)\n\n\n[JSON Schema](https://github.com/datacontract/datacontract-specification/blob/main/datacontract.schema.json) of the Data Contract Specification.\n\n### Data Contract Object\n\nThis is the root document.\n\nIt is _RECOMMENDED_ that the root document be named: `datacontract.yaml`.\n\n| Field | Type | Description |\n|---------------------------|------------------------------------------------------|----------------------------------------------------------------------------------------------------------|\n| dataContractSpecification | `string` | REQUIRED. Specifies the Data Contract Specification being used. |\n| id | `string` | REQUIRED. An organization-wide unique technical identifier, such as a UUID, URN, slug, string, or number |\n| info | [Info Object](#info-object) | REQUIRED. Specifies the metadata of the data contract. May be used by tooling. |\n| servers | Map[string, [Server Object](#server-object)] | Specifies the servers of the data contract. |\n| terms | [Terms Object](#terms-object) | Specifies the terms and conditions of the data contract. |\n| models | Map[string, [Model Object](#model-object)] | Specifies the logical data model. |\n| definitions | Map[string, [Definition Object](#definition-object)] | Specifies definitions. |\n| schema | [Schema Object](#schema-object) | Specifies the physical schema. The specification supports different schema format. |\n| examples | Array of [Example Objects](#example-object) | Specifies example data sets for the data model. The specification supports different example types. |\n| quality | [Quality Object](#quality-object) | Specifies the quality attributes and checks. The specification supports different quality check DSLs. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n\n### Info Object\n\nMetadata and life cycle information about the data contract.\n\n\n| Field | Type | Description |\n|---------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| title | `string` | REQUIRED. The title of the data contract. |\n| version | `string` | REQUIRED. The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version). |\n| description | `string` | A description of the data contract. |\n| owner | `string` | The owner or team responsible for managing the data contract and providing the data. |\n| contact | [Contact Object](#contact-object) | Contact information for the data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Contact Object\n\nContact information for the data contract.\n\n| Field | Type | Description |\n|-------|----------|-------------------------------------------------------------------------------------------------------|\n| name | `string` | The identifying name of the contact person/organization. |\n| url | `string` | The URL pointing to the contact information. This _MUST_ be in the form of a URL. |\n| email | `string` | The email address of the contact person/organization. This _MUST_ be in the form of an email address. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n### Server Object\n\nThe fields are dependent on the defined type.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `s3`, `redshift`, `snowflake`, `databricks`, `kafka` |\n| description | `string` | An optional string describing the server. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### BigQuery Server Object\n\n| Field | Type | Description |\n|---------|----------|-------------|\n| type | `string` | `bigquery` |\n| project | `string` | |\n| dataset | `string` | |\n\n#### S3 Server Object\n\n| Field | Type | Description |\n|----------|----------|--------------------------------|\n| type | `string` | `s3` |\n| location | `string` | S3 URL, starting with `s3://` |\n\nExample:\n\n```yaml\nservers:\n production:\n type: s3\n location: s3://acme-orders-prod/orders/\n```\n\n\n#### Redshift Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `redshift` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Snowflake Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `snowflake` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Databricks Server Object\n\n| Field | Type | Description |\n|----------|----------|--------------|\n| type | `string` | `databricks` |\n| share | `string` | |\n\n#### Kafka Server Object\n\n| Field | Type | Description |\n|-------|----------|-------------|\n| type | `string` | `kafka` |\n| host | `string` | |\n| topic | `string` | |\n\n\n\n### Terms Object\n\nThe terms and conditions of the data contract.\n\n| Field | Type | Description |\n|----------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| usage | `string` | The usage describes the way the data is expected to be used. Can contain business and technical information. |\n| limitations | `string` | The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for. |\n| billing | `string` | The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use. |\n| noticePeriod | `string` | The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., `P3M` for a period of three months. |\n\n\n### Model Object\n\nThe Model Object describes the structure and semantics of a data model, such as tables, views, or structured files.\n\nThe name of the data model (table name) is defined by the key that refers to this Model Object.\n\n| Field | Type | Description |\n|-------------|----------------------------------------------|-----------------------------------------------------------------------|\n| type | `string` | The type of the model. Examples: `table`, `object`. Default: `table`. |\n| description | `string` | An optional string describing the data model. |\n| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |\n\n\n\n### Field Object\n\nThe Field Objects describes one field (column, property, nested field) of a data model.\n\n| Field | Type | Description |\n|----------------|--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | [Data Type](#data-types) | The logical data type of the field. |\n| description | `string` | An optional string describing the semantic of the data in this field. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). | \n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. Examples may be: `sensitive`, `restricted`, `internal`, `public`. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| $ref | `string` | A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition. |\n\n\n### Definition Object\n\nThe Definition Object includes a clear and concise explanations of syntax, semantic, and classification of a business object in a given domain.\nIt serves as a reference for a common understanding of terminology, ensure consistent usage and to identify join-able fields.\nModels fields can refer to definitions using the `$ref` field to link to existing definitions and avoid duplicate documentations.\n\n| Field | Type | Description |\n|----------------|--------------------------|----------------------------------------------------------------------------------------------------------------------|\n| domain | `string` | The domain in which this definition is valid. Default: `global`. |\n| name | `string` | The technical name of this definition. |\n| title | `string` | The business name of this definition. |\n| type | [Data Type](#data-types) | The logical data type |\n| description | `string` | Clear and concise explanations related to the domain |\n| example | `string` | An example value. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). |\n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n\n\n### Schema Object\n\nThe schema of the data contract describes the physical schema. \nThe type of the schema depends on the data platform.\n\n| Field | Type | Description |\n| ----- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `dbt`, `bigquery`, `json-schema`, `sql-ddl`, `avro`, `protobuf`, `custom` |\n| specification | [dbt Schema Object](#dbt-schema-object) \\|
[BigQuery Schema Object](#bigquery-schema-object) \\|
[JSON Schema Schema Object](#bigquery-schema-object) \\|
[SQL DDL Schema Object](#sql-ddl-schema-object) \\|
`string` | REQUIRED. The specification of the schema. The schema specification can be encoded as a string or as inline YAML. |\n\n\n#### dbt Schema Object\n\nhttps://docs.getdbt.com/reference/model-properties\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: dbt\n specification:\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\nExample (string):\n\n```yaml\nschema:\n type: dbt\n specification: |-\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\n#### BigQuery Schema Object\n\nThe schema structure is defined by the [Google BigQuery Table](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#resource:-table) object. You can extract such a Table object via the [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) endpoint.\n\nInstead of providing a single Table object, you can also provide an array of such objects. Be aware that [tables.list](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list) only returns a subset of the full Table object. You need to call every Table object via [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) to get the full Table object, including the actual schema.\n\nLearn more: [Google BigQuery REST Reference v2](https://cloud.google.com/bigquery/docs/reference/rest)\n\n\n\nExample:\n\n```yaml\nschema:\n type: bigquery\n specification: |-\n {\n \"tableReference\": {\n \"projectId\": \"my-project\",\n \"datasetId\": \"my_dataset\",\n \"tableId\": \"my_table\"\n },\n \"description\": \"This is a description\",\n \"type\": \"TABLE\",\n \"schema\": {\n \"fields\": [\n {\n \"name\": \"name\",\n \"type\": \"STRING\",\n \"mode\": \"NULLABLE\",\n \"description\": \"This is a description\"\n }\n ]\n }\n }\n```\n\n#### JSON Schema Schema Object\n\nJSON Schema can be defined as JSON or rendered as YAML, following the [OpenAPI Schema Object dialect](https://spec.openapis.org/oas/v3.1.0#properties)\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: json-schema\n specification:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: object\n properties:\n order_id:\n type: string\n description: Primary key of the orders table\n order_timestamp:\n type: string\n format: date-time\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total:\n type: integer\n description: Total amount of the order in the smallest monetary unit (e.g., cents).\n line_items:\n type: object\n properties:\n lines_item_id:\n type: string\n description: Primary key of the lines_item_id table\n order_id:\n type: string\n description: Foreign key to the orders table\n sku:\n type: string\n description: The purchased article number\n```\n\nExample (string):\n\n```yaml\nschema:\n type: json-schema\n specification: |-\n {\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"orders\": {\n \"type\": \"object\",\n \"description\": \"One record per order. Includes cancelled and deleted orders.\",\n \"properties\": {\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the orders table\"\n },\n \"order_timestamp\": {\n \"type\": \"string\",\n \"format\": \"date-time\",\n \"description\": \"The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\"\n },\n \"order_total\": {\n \"type\": \"integer\",\n \"description\": \"Total amount of the order in the smallest monetary unit (e.g., cents).\"\n }\n },\n \"required\": [\"order_id\", \"order_timestamp\", \"order_total\"]\n },\n \"line_items\": {\n \"type\": \"object\",\n \"properties\": {\n \"lines_item_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the lines_item_id table\"\n },\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Foreign key to the orders table\"\n },\n \"sku\": {\n \"type\": \"string\",\n \"description\": \"The purchased article number\"\n }\n },\n \"required\": [\"lines_item_id\", \"order_id\", \"sku\"]\n }\n },\n \"required\": [\"orders\", \"line_items\"]\n }\n```\n\n#### SQL DDL Schema Object\n\nClassical SQL DDLs can be used to describe the structure.\n\n\nExample (string):\n\n```yaml\nschema:\n type: sql-ddl\n specification: |-\n -- One record per order. Includes cancelled and deleted orders.\n CREATE TABLE orders (\n order_id TEXT PRIMARY KEY, -- Primary key of the orders table\n order_timestamp TIMESTAMPTZ NOT NULL, -- The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total INTEGER NOT NULL -- Total amount of the order in the smallest monetary unit (e.g., cents)\n );\n \n -- The items that are part of an order\n CREATE TABLE line_items (\n lines_item_id TEXT PRIMARY KEY, -- Primary key of the lines_item_id table\n order_id TEXT REFERENCES orders(order_id), -- Foreign key to the orders table\n sku TEXT NOT NULL -- The purchased article number\n );\n\n```\n\n### Example Object\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `csv`, `json`, `yaml`, `custom` |\n| description | `string` | An optional string describing the example. |\n| model | `string` | The reference to the model in the schema, e.g. a table name. | \n| data | `string` | Example data for this model. |\n\nExample:\n\n```yaml\nexamples:\n- type: csv\n model: orders\n data: |-\n order_id,order_timestamp,order_total\n \"1001\",\"2023-09-09T08:30:00Z\",2500\n \"1002\",\"2023-09-08T15:45:00Z\",1800\n \"1003\",\"2023-09-07T12:15:00Z\",3200\n \"1004\",\"2023-09-06T19:20:00Z\",1500\n \"1005\",\"2023-09-05T10:10:00Z\",4200\n \"1006\",\"2023-09-04T14:55:00Z\",2800\n \"1007\",\"2023-09-03T21:05:00Z\",1900\n \"1008\",\"2023-09-02T17:40:00Z\",3600\n \"1009\",\"2023-09-01T09:25:00Z\",3100\n \"1010\",\"2023-08-31T22:50:00Z\",2700\n```\n\n### Quality Object\n\nThe quality object contains quality attributes and checks.\n\n| Field | Type | Description |\n| ----- |-------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `SodaCL`, `montecarlo`, `custom` |\n| specification | [SodaCL Quality Object](#sodacl-quality-object) \\|
[Monte Carlo Schema Object](#monte-carlo-quality-object) \\|
`string` | REQUIRED. The specification of the quality attributes. The quality specification can be encoded as a string or as inline YAML. |\n\n\n#### SodaCL Quality Object\n\nQuality attributes in [Soda Checks Language](https://docs.soda.io/soda-cl/soda-cl-overview.html).\n\nThe `specification` represents the content of a `checks.yml` file.\n\nExample (inline):\n\n```yaml\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, dbt-tests, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - row_count > 0\n - duplicate_count(order_id) = 0\n checks for line_items:\n - row_count > 0\n```\n\nExample (string):\n\n```yaml\nquality:\n type: SodaCL\n specification: |-\n checks for search_queries:\n - freshness(search_timestamp) < 1d\n - row_count > 100000\n - missing_count(search_query) = 0\n```\n\n#### Monte Carlo Quality Object\n\nQuality attributes defined as Monte Carlos [Monitors as Code](https://docs.getmontecarlo.com/docs/monitors-as-code).\n\nThe `specification` represents the content of a `montecarlo.yml` file.\n\nExample (string):\n\n```yaml\nquality:\n type: montecarlo\n specification: |-\n montecarlo:\n field_health:\n - table: project:dataset.table_name\n timestamp_field: created\n dimension_tracking:\n - table: project:dataset.table_name\n timestamp_field: created\n field: order_status\n```\n\n### Data Types\n\nThe following data types are supported for model fields and definitions:\n\n- Unicode character sequence: `string`, `text`, `varchar`\n- Any numeric type, either integers or floating point numbers: `number`, `decimal`, `numeric`\n- 32-bit signed integer: `int`, `integer`\n- 64-bit signed integer: `long`, `bigint`\n- Single precision (32-bit) IEEE 754 floating-point number: `float`\n- Double precision (64-bit) IEEE 754 floating-point number: `double`\n- Binary value: `boolean`\n- Timestamp with timezone: `timestamp`, `timestamp_tz`\n- Timestamp with no timezone: `timestamp_ntz`\n- Date with no time information: `date`\n- Array: `array`\n- Sequence of 8-bit unsigned bytes: `bytes`\n- Complex type: `object`, `record`, `struct`\n- No value: `null`\n\n### Specification Extensions\n\nWhile the Data Contract Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points.\n\nA custom fields can be added with any name. The value can be null, a primitive, an array or an object. \n\n### Design Principles\n\nThe Data Contract Specification follows these design principles:\n\n- A free, open, and open-sourced standard\n- Follow OpenAPI and AsyncAPI conventions so that it feels immediately familiar \n- Support contract-first approaches\n- Support code-first approaches\n- Support tooling by being machine-readable\n\nTooling\n---\n- [Data Contract CLI](https://github.com/datacontract/cli) is a free CLI tool to help you create, develop, and maintain your data contracts.\n- [Data Contract Studio](https://studio.datacontract.com/) is a free web tool to develop and share data contracts.\n- [Data Mesh Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data products and data contracts. It supports the data contract specification and allows the user to import or export data contracts using this specification.\n\n\nOther Data Contract Specifications\n---\n- [AIDA User Group's Open Data Contract Standard](https://github.com/AIDAUserGroup/open-data-contract-standard)\n- [PayPal's Data Contract Template](https://github.com/paypal/data-contract-template/blob/main/docs/README.md)\n\nLiterature\n---\n- [Driving Data Quality with Data Contracts](https://www.amazon.com/dp/B0C37FPH3D) by Andrew Jones\n\nAuthors\n---\nThe Data Contract Specification was originally created by [Jochen Christ](https://www.linkedin.com/in/jochenchrist/) and [Dr. Simon Harrer](https://www.linkedin.com/in/simonharrer/), and is currently maintained by them.\n\n\nContributing\n---\nContributions are welcome! Please open an issue or a pull request.\n\nLicense\n---\n[MIT License](LICENSE)\n\n\n\n" }, { "path": "versions/0.9.1/datacontract.init.yaml", "content": "dataContractSpecification: 0.9.1\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# my-stage:\n# type: bigquery\n# project:\n# dataset:\n\n#servers:\n# my-stage:\n# type: s3\n# location: s3://\n\n#servers:\n# my-stage:\n# type: redshift\n# account:\n# database:\n# schema:\n\n#servers:\n# my-stage:\n# type: snowflake\n# account:\n# database:\n# schema:\n\n#servers:\n# my-stage:\n# type: databricks\n# share:\n\n#servers:\n# my-stage:\n# type: kafka\n# host:\n# topic:\n\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n### models\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type: \n# description:\n\n### definitions\n# definitions: \n# my_field: \n# domain: \n# name: \n# title: \n# type: \n# description: \n# example: \n# pii:\n# classification: \n\n### schema\n\n#schema:\n# type: dbt\n# specification:\n# version:\n# models:\n# - name:\n# description:\n# columns:\n# - name:\n# type:\n# description:\n# tests:\n\n#schema:\n# type: dbt\n# specification: |-\n# version:\n# models:\n# - name:\n# description:\n# columns:\n# - name:\n# type:\n# description:\n# tests:\n\n#schema:\n# type: dbt\n# specification: \"$ref: model.yaml\"\n\n#schema:\n# type: bigquery\n# specification: |-\n# {\n# \"tableReference\": {\n# \"projectId\": \"my-project\",\n# \"datasetId\": \"my_dataset\",\n# \"tableId\": \"my_table\"\n# },\n# \"description\": \"This is a description\",\n# \"type\": \"TABLE\",\n# \"schema\": {\n# \"fields\": [\n# {\n# \"name\": \"name\",\n# \"type\": \"STRING\",\n# \"mode\": \"NULLABLE\",\n# \"description\": \"This is a description\"\n# }\n# ]\n# }\n# }\n\n#schema:\n# type: json-schema\n# specification:\n# my-table:\n# description:\n# type: object\n# properties:\n# id:\n# type: string\n# description:\n\n#schema:\n# type: json-schema\n# specification: |-\n# {\n# \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n# \"type\": \"object\",\n# \"properties\": {\n# \"my_table\": {\n# \"type\": \"object\",\n# \"description\": \"\",\n# \"properties\": {\n# \"id\": {\n# \"type\": \"string\",\n# \"description\": \"\"\n# },\n# \"required\": [\"id\"]\n# }\n# },\n# \"required\": [\"my-table\"]\n# }\n\n#schema:\n# type: sql-ddl\n# specification: |-\n# CREATE TABLE my_table (\n# id TEXT PRIMARY KEY\n# );\n\n#schema:\n# type: avro\n# specification:\n# User:\n# type: record\n# name: MyTable\n# fields:\n# - name: id\n# type: string\n\n#schema:\n# type: avro\n# specification: |-\n# {\n# \"type\": \"record\",\n# \"name\": \"MyTable\",\n# \"fields\": [\n# {\n# \"name\": \"name\",\n# \"type\": \"string\"\n# }\n# ]\n# }\n\n#schema:\n# type: protobuf\n# specification: |-\n# message MyTable {\n# string id = 1;\n# }\n\n#schema:\n# type: custom\n# specification:\n\n\n### examples\n\n#examples:\n# - type: csv\n# model: my_table\n# data: |-\n# id,timestamp,amount\n# \"1001\",\"2023-09-09T08:30:00Z\",2500\n# \"1002\",\"2023-09-08T15:45:00Z\",1800\n#\n#examples:\n# - type: csv\n# model: my_table\n# data: \"$ref: data.csv\"\n\n#examples:\n# - type: json\n# model: my_table\n# data: |-\n# [\n# {\n# \"id\": \"1001\",\n# \"timestamp\": \"2023-09-09T08:30:00Z\",\n# \"amount\": 2500\n# },\n# {\n# \"id\": \"1002\",\n# \"timestamp\": \"2023-09-08T15:45:00Z\",\n# \"amount\": 1800\n# }\n# ]\n\n#examples:\n# - type: yaml\n# model: my_table\n# data:\n# - id: 1001\n# timestamp: 2023-09-09T08:30:00Z\n# amount: 2500\n# - id: 1002\n# timestamp: 2023-09-08T15:45:00Z\n# amount: 1800\n\n#examples:\n# - type: custom\n# model: my_table\n# data: |-\n\n\n### quality\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_table:\n# - duplicate_count(order_id) = 0\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_table: |-\n# - duplicate_count(id) = 0\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_table: \"$ref: checks.yaml\"\n\n#quality:\n# type: montecarlo\n# specification: |-\n# montecarlo:\n# field_health:\n# - table: my_project:my_dataset.my_table\n# fields:\n# - id\n# - timestamp\n# - amount\n# timestamp_field: timestamp\n\n#quality:\n# type: custom\n# specification: |-\n" }, { "path": "versions/0.9.1/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"enum\": [\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\"\n }\n },\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"bigquery\",\n \"BigQuery\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"project\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"project\",\n \"dataset\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"s3\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"An optional string describing the server. Must be in the form of a URL.\"\n }\n },\n \"required\": [\n \"type\",\n \"location\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"redshift\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"snowflake\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"databricks\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"share\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"share\"\n ]\n },\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"kafka\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"type\",\n \"host\",\n \"topic\"\n ]\n }\n ]\n },\n \"description\": \"Information about the servers.\"\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n }\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, object. Default: table.\",\n \"type\": \"string\",\n \"default\": \"table\"\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"properties\": {\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\", \"decimal\", \"numeric\",\n \"int\", \"integer\",\n \"long\", \"bigint\",\n \"float\",\n \"double\",\n \"string\", \"text\", \"varchar\",\n \"boolean\",\n \"timestamp\", \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"array\",\n \"object\", \"record\", \"struct\",\n \"bytes\",\n \"null\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\"sensitive\", \"restricted\", \"internal\", \"public\"]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n }\n }\n }\n }\n }\n }\n },\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"dbt\",\n \"bigquery\",\n \"json-schema\",\n \"sql-ddl\",\n \"avro\",\n \"protobuf\",\n \"custom\"\n ],\n \"description\": \"The type of the schema. Typical values are: dbt, bigquery, json-schema, sql-ddl, avro, protobuf, custom.\"\n },\n \"specification\": {\n \"anyOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the schema as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the schema as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The schema of the data contract describes the syntax and semantics of provided data sets. It supports different schema types.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"csv\",\n \"json\",\n \"yaml\",\n \"custom\"\n ],\n \"description\": \"The type of the example data. Well-known types are: csv, json, yaml, custom.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the example.\"\n },\n \"model\": {\n \"type\": \"string\",\n \"description\": \"The reference to the model in the schema, e.g., a table name.\"\n },\n \"data\": {\n \"type\": \"string\",\n \"description\": \"Example data for this model.\"\n }\n },\n \"required\": [\n \"type\",\n \"data\"\n ]\n },\n \"description\": \"The Examples Object is an array of Example Objects.\"\n },\n \"quality\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"SodaCL\",\n \"montecarlo\",\n \"custom\"\n ],\n \"description\": \"The type of the quality check. Typical values are: SodaCL, montecarlo, custom.\"\n },\n \"specification\": {\n \"anyOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the quality attributes as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the quality attributes as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The quality object contains quality attributes and checks.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ]\n}\n" }, { "path": "versions/0.9.2/README.md", "content": "# Data Contract Specification\n\n\n \"Stars\"\n\"Slack\n\n![datacontract.png](images/datacontract.png)\n\nData contracts bring data providers and data consumers together.\n\nA _data contract_ is a document that defines the structure, format, semantics, quality, and terms of use for exchanging data between a data provider and their consumers.\nA data contract is implemented by a data product's output port or other data technologies.\nData contracts can also be used for the input port to specify the expectations of data dependencies and verify given guarantees.\n\nThe _data contract specification_ defines a YAML format to describe attributes of provided data sets.\nIt is data platform neutral and can be used with any data platform, such as AWS S3, Google BigQuery, Microsoft Fabric, Databricks, and Snowflake.\nThe data contract specification is an open initiative to define a common data contract format.\nIt follows [OpenAPI](https://www.openapis.org/) and [AsyncAPI](https://www.asyncapi.com/) conventions.\n\nData contracts come into play when data is exchanged between different teams or organizational units, such as in a [data mesh architecture](https://www.datamesh-architecture.com/).\nFirst, and foremost, data contracts are a communication tool to express a common understanding of how data should be structured and interpreted.\nThey make semantic and quality expectations explicit.\nThey are often created collaboratively in [workshops](/workshop) together with data providers and data consumers.\nLater in development and production, they also serve as the basis for code generation, testing, schema validations, quality checks, monitoring, access control, and computational governance policies.\n\nThe specification comes along with the [Data Contract CLI](https://github.com/datacontract/cli), an open-source tool to develop, validate, and enforce data contracts.\n\n_Note: The term \"data contract\" refers to a specification that is usually owned by the data provider and thus does not align with a \"contract\" in a legal sense as a mutual agreement between two parties.\nThe term \"contract\" may be somewhat misleading, but it is how it is used in practice.\nThe mutual agreement between one data provider and one data consumer is the \"data usage agreement\" that refers to a data contract.\nData usage agreements have a defined lifecycle, start/end date, and help the data provider to track who accesses their data and for which purposes._\n\nVersion\n---\n\n0.9.2 ([Changelog](CHANGELOG.md))\n\nExample\n---\n\n[![Open in Data Contract Studio](https://img.shields.io/badge/open%20in-Data%20Contract%20Studio-blue)](https://studio.datacontract.com/)\n\n```yaml\ndataContractSpecification: 0.9.2\nid: urn:datacontract:checkout:orders-latest\ninfo:\n title: Orders Latest\n version: 1.0.0\n description: |\n Successful customer orders in the webshop. \n All orders since 2020-01-01. \n Orders with their line items are in their current state (no history included). \n owner: Checkout Team\n contact:\n name: John Doe (Data Product Owner)\n url: https://teams.microsoft.com/l/channel/example/checkout\nservers:\n production:\n type: s3\n location: s3://datacontract-example-orders-latest/data/{model}/*.json\n format: json\n delimiter: new_line\nterms:\n usage: >\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: >\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table\n fields:\n order_id:\n $ref: '#/definitions/order_id'\n required: true\n unique: true\n order_timestamp:\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n type: timestamp\n required: true\n order_total:\n description: Total amount the smallest monetary unit (e.g., cents).\n type: long\n required: true\n customer_id:\n description: Unique identifier for the customer.\n type: text\n minLength: 10\n maxLength: 20\n customer_email_address:\n description: The email address, as entered by the customer. The email address was not verified.\n type: text\n format: email\n required: true\n line_items:\n description: A single article that is part of an order.\n type: table\n fields:\n lines_item_id:\n type: text\n description: Primary key of the lines_item_id table\n required: true\n unique: true\n order_id:\n $ref: '#/definitions/order_id'\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\ndefinitions:\n order_id:\n domain: checkout\n name: order_id\n title: Order ID\n type: text\n format: uuid\n description: An internal ID that identifies an order in the online shop.\n example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n sku:\n domain: inventory\n name: sku\n title: Stock Keeping Unit\n type: text\n pattern: ^[A-Za-z0-9]{8,14}$\n example: \"96385074\"\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article. \n It is typically associated with an article's barcode, such as the EAN/GTIN.\nexamples:\n - type: csv # csv, json, yaml, custom\n model: orders\n data: |- # expressed as string or inline yaml or via \"$ref: data.csv\"\n order_id,order_timestamp,order_total,customer_id,customer_email_address\n \"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\"\n \"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\"\n \"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\"\n \"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\"\n \"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\"\n \"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\"\n \"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\"\n \"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\"\n \"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\"\n \"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\"\n - type: csv\n model: line_items\n data: |-\n lines_item_id,order_id,sku\n \"LI-1\",\"1001\",\"5901234123457\"\n \"LI-2\",\"1001\",\"4001234567890\"\n \"LI-3\",\"1002\",\"5901234123457\"\n \"LI-4\",\"1002\",\"2001234567893\"\n \"LI-5\",\"1003\",\"4001234567890\"\n \"LI-6\",\"1003\",\"5001234567892\"\n \"LI-7\",\"1004\",\"5901234123457\"\n \"LI-8\",\"1005\",\"2001234567893\"\n \"LI-9\",\"1005\",\"5001234567892\"\n \"LI-10\",\"1005\",\"6001234567891\"\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - freshness(order_timestamp) < 24h\n - row_count >= 5000\n - duplicate_count(order_id) = 0\n checks for line_items:\n - values in (order_id) must exist in orders (order_id)\n - row_count >= 5000\n```\n\nData Contract CLI\n---\n\nThe [Data Contract CLI](https://cli.datacontract.com) is a command line tool and Python library to lint, test, import and export data contracts.\n\nHere is short example how to verify that your actual dataset matches the data contract:\n\n```bash\npip3 install datacontract-cli\ndatacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nor, if you prefer Docker:\n```bash\ndocker run datacontract/cli test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nThe Data Contract contains all required information to verify data:\n\n- The _servers_ block has the connection details to the actual data set.\n- The _models_ define the syntax, formats, and constraints.\n- The _quality_ defined further quality checks.\n\nThe Data Contract CLI chooses the appropriate engine, formulates test cases, connects to the server, and executes the tests, based on the server type.\n\nMore information and configuration options on [cli.datacontract.com](https://cli.datacontract.com).\n\n\nSpecification\n---\n\n![The eight major categories in the data contract specification](images/categories.png)\n\n- [Data Contract Object](#data-contract-object)\n- [Info Object](#info-object)\n- [Contact Object](#contact-object)\n- [Server Object](#server-object)\n- [Terms Object](#terms-object)\n- [Model Object](#model-object)\n- [Field Object](#field-object)\n- [Definition Object](#definition-object)\n- [Schema Object](#schema-object)\n- [Example Object](#example-object)\n- [Quality Object](#quality-object)\n- [Data Types](#data-types)\n- [Specification Extensions](#specification-extensions)\n\n\n[JSON Schema](https://github.com/datacontract/datacontract-specification/blob/main/datacontract.schema.json) of the Data Contract Specification.\n\n### Data Contract Object\n\nThis is the root document.\n\nIt is _RECOMMENDED_ that the root document be named: `datacontract.yaml`.\n\n| Field | Type | Description |\n|---------------------------|------------------------------------------------------|----------------------------------------------------------------------------------------------------------|\n| dataContractSpecification | `string` | REQUIRED. Specifies the Data Contract Specification being used. |\n| id | `string` | REQUIRED. An organization-wide unique technical identifier, such as a UUID, URN, slug, string, or number |\n| info | [Info Object](#info-object) | REQUIRED. Specifies the metadata of the data contract. May be used by tooling. |\n| servers | Map[string, [Server Object](#server-object)] | Specifies the servers of the data contract. |\n| terms | [Terms Object](#terms-object) | Specifies the terms and conditions of the data contract. |\n| models | Map[string, [Model Object](#model-object)] | Specifies the logical data model. |\n| definitions | Map[string, [Definition Object](#definition-object)] | Specifies definitions. |\n| schema | [Schema Object](#schema-object) | Specifies the physical schema. The specification supports different schema format. |\n| examples | Array of [Example Objects](#example-object) | Specifies example data sets for the data model. The specification supports different example types. |\n| quality | [Quality Object](#quality-object) | Specifies the quality attributes and checks. The specification supports different quality check DSLs. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n\n### Info Object\n\nMetadata and life cycle information about the data contract.\n\n\n| Field | Type | Description |\n|---------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| title | `string` | REQUIRED. The title of the data contract. |\n| version | `string` | REQUIRED. The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version). |\n| description | `string` | A description of the data contract. |\n| owner | `string` | The owner or team responsible for managing the data contract and providing the data. |\n| contact | [Contact Object](#contact-object) | Contact information for the data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Contact Object\n\nContact information for the data contract.\n\n| Field | Type | Description |\n|-------|----------|-------------------------------------------------------------------------------------------------------|\n| name | `string` | The identifying name of the contact person/organization. |\n| url | `string` | The URL pointing to the contact information. This _MUST_ be in the form of a URL. |\n| email | `string` | The email address of the contact person/organization. This _MUST_ be in the form of an email address. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n### Server Object\n\nThe fields are dependent on the defined type.\n\n| Field | Type | Description |\n|-------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `s3`, `redshift`, `snowflake`, `databricks`, `postgres`, `kafka`, `pubsub`, `local` |\n| description | `string` | An optional string describing the server. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### BigQuery Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `bigquery` |\n| project | `string` | The GCP project name. |\n| dataset | `string` | |\n\n#### S3 Server Object\n\n| Field | Type | Description |\n|----------|----------|--------------------------------|\n| type | `string` | `s3` |\n| location | `string` | S3 URL, starting with `s3://` |\n| endpointUrl | `string` | The server endpoint for S3-compatible servers, such as `https://minio.example.com` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\nExample:\n\n```yaml\nservers:\n production:\n type: s3\n location: s3://acme-orders-prod/orders/\n```\n\n\n#### Redshift Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `redshift` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Snowflake Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `snowflake` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Databricks Server Object\n\n| Field | Type | Description |\n|---------|----------|---------------------------------------------------------------------|\n| type | `string` | `databricks` |\n| host | `string` | The Databricks host, e.g., `dbc-abcdefgh-1234.cloud.databricks.com` |\n| catalog | `string` | The name of the Hive or Unity catalog |\n| schema | `string` | The schema name in the catalog |\n\n#### Postgres Server Object\n\n| Field | Type | Description |\n|----------|-----------|---------------------------------------------------------|\n| type | `string` | `postgres` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server |\n| database | `string` | The name of the database, e.g., `postgres`. |\n| schema | `string` | The name of the schema in the database, e.g., `public`. |\n\n\n#### Kafka Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kafka` |\n| host | `string` | The bootstrap server of the kafka cluster. |\n| topic | `string` | The topic name. |\n| format | `string` | The format of the message. Examples: json, avro, protobuf. Default: json. |\n\n#### Pub/Sub Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `pubsub` |\n| project | `string` | The GCP project name. |\n| topic | `string` | The topic name. |\n\n#### Local Server Object\n\n| Field | Type | Description |\n|--------|----------|-------------------------------------------------------------------------------------|\n| type | `string` | `local` |\n| path | `string` | The relative or absolute path to the data file(s), such as `./folder/data.parquet`. |\n| format | `string` | The format of the file(s), such as `parquet`, `csv`, or `json`. |\n\n\n\n### Terms Object\n\nThe terms and conditions of the data contract.\n\n| Field | Type | Description |\n|----------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| usage | `string` | The usage describes the way the data is expected to be used. Can contain business and technical information. |\n| limitations | `string` | The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for. |\n| billing | `string` | The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use. |\n| noticePeriod | `string` | The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., `P3M` for a period of three months. |\n\n\n### Model Object\n\nThe Model Object describes the structure and semantics of a data model, such as tables, views, or structured files.\n\nThe name of the data model (table name) is defined by the key that refers to this Model Object.\n\n| Field | Type | Description |\n|-------------|----------------------------------------------|-------------------------------------------------------------------------------|\n| type | `string` | The type of the model. Examples: `table`, `view`, `object`. Default: `table`. |\n| description | `string` | An optional string describing the data model. |\n| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |\n\n\n\n### Field Object\n\nThe Field Objects describes one field (column, property, nested field) of a data model.\n\n| Field | Type | Description |\n|------------------|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the semantic of the data in this field. |\n| type | [Data Type](#data-types) | The logical data type of the field. |\n| required | `boolean` | An indication, if this field must contain a value and may not be null. Default: `false` |\n| primary | `boolean` | If this field is a primary key. Default: `false` |\n| references | `string` | The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship. |\n| unique | `boolean` | An indication, if the value must be unique within the model. Default: `false` |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| example | `string` | An example value. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). | \n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. Examples may be: `sensitive`, `restricted`, `internal`, `public`. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| $ref | `string` | A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition. |\n\n\n### Definition Object\n\nThe Definition Object includes a clear and concise explanations of syntax, semantic, and classification of a business object in a given domain.\nIt serves as a reference for a common understanding of terminology, ensure consistent usage and to identify join-able fields.\nModels fields can refer to definitions using the `$ref` field to link to existing definitions and avoid duplicate documentations.\n\n| Field | Type | Description |\n|------------------|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| domain | `string` | The domain in which this definition is valid. Default: `global`. |\n| name | `string` | The technical name of this definition. |\n| title | `string` | The business name of this definition. |\n| description | `string` | Clear and concise explanations related to the domain |\n| type | [Data Type](#data-types) | The logical data type |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| example | `string` | An example value. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). |\n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n\n\n### Schema Object\n\nThe schema of the data contract describes the physical schema.\nThe type of the schema depends on the data platform.\n\n| Field | Type | Description |\n| ----- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `dbt`, `bigquery`, `json-schema`, `sql-ddl`, `avro`, `protobuf`, `custom` |\n| specification | [dbt Schema Object](#dbt-schema-object) \\|
[BigQuery Schema Object](#bigquery-schema-object) \\|
[JSON Schema Schema Object](#bigquery-schema-object) \\|
[SQL DDL Schema Object](#sql-ddl-schema-object) \\|
`string` | REQUIRED. The specification of the schema. The schema specification can be encoded as a string or as inline YAML. |\n\n\n#### dbt Schema Object\n\nhttps://docs.getdbt.com/reference/model-properties\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: dbt\n specification:\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\nExample (string):\n\n```yaml\nschema:\n type: dbt\n specification: |-\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\n#### BigQuery Schema Object\n\nThe schema structure is defined by the [Google BigQuery Table](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#resource:-table) object. You can extract such a Table object via the [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) endpoint.\n\nInstead of providing a single Table object, you can also provide an array of such objects. Be aware that [tables.list](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list) only returns a subset of the full Table object. You need to call every Table object via [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) to get the full Table object, including the actual schema.\n\nLearn more: [Google BigQuery REST Reference v2](https://cloud.google.com/bigquery/docs/reference/rest)\n\n\n\nExample:\n\n```yaml\nschema:\n type: bigquery\n specification: |-\n {\n \"tableReference\": {\n \"projectId\": \"my-project\",\n \"datasetId\": \"my_dataset\",\n \"tableId\": \"my_table\"\n },\n \"description\": \"This is a description\",\n \"type\": \"TABLE\",\n \"schema\": {\n \"fields\": [\n {\n \"name\": \"name\",\n \"type\": \"STRING\",\n \"mode\": \"NULLABLE\",\n \"description\": \"This is a description\"\n }\n ]\n }\n }\n```\n\n#### JSON Schema Schema Object\n\nJSON Schema can be defined as JSON or rendered as YAML, following the [OpenAPI Schema Object dialect](https://spec.openapis.org/oas/v3.1.0#properties)\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: json-schema\n specification:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: object\n properties:\n order_id:\n type: string\n description: Primary key of the orders table\n order_timestamp:\n type: string\n format: date-time\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total:\n type: integer\n description: Total amount of the order in the smallest monetary unit (e.g., cents).\n line_items:\n type: object\n properties:\n lines_item_id:\n type: string\n description: Primary key of the lines_item_id table\n order_id:\n type: string\n description: Foreign key to the orders table\n sku:\n type: string\n description: The purchased article number\n```\n\nExample (string):\n\n```yaml\nschema:\n type: json-schema\n specification: |-\n {\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"orders\": {\n \"type\": \"object\",\n \"description\": \"One record per order. Includes cancelled and deleted orders.\",\n \"properties\": {\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the orders table\"\n },\n \"order_timestamp\": {\n \"type\": \"string\",\n \"format\": \"date-time\",\n \"description\": \"The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\"\n },\n \"order_total\": {\n \"type\": \"integer\",\n \"description\": \"Total amount of the order in the smallest monetary unit (e.g., cents).\"\n }\n },\n \"required\": [\"order_id\", \"order_timestamp\", \"order_total\"]\n },\n \"line_items\": {\n \"type\": \"object\",\n \"properties\": {\n \"lines_item_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the lines_item_id table\"\n },\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Foreign key to the orders table\"\n },\n \"sku\": {\n \"type\": \"string\",\n \"description\": \"The purchased article number\"\n }\n },\n \"required\": [\"lines_item_id\", \"order_id\", \"sku\"]\n }\n },\n \"required\": [\"orders\", \"line_items\"]\n }\n```\n\n#### SQL DDL Schema Object\n\nClassical SQL DDLs can be used to describe the structure.\n\n\nExample (string):\n\n```yaml\nschema:\n type: sql-ddl\n specification: |-\n -- One record per order. Includes cancelled and deleted orders.\n CREATE TABLE orders (\n order_id TEXT PRIMARY KEY, -- Primary key of the orders table\n order_timestamp TIMESTAMPTZ NOT NULL, -- The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total INTEGER NOT NULL -- Total amount of the order in the smallest monetary unit (e.g., cents)\n );\n \n -- The items that are part of an order\n CREATE TABLE line_items (\n lines_item_id TEXT PRIMARY KEY, -- Primary key of the lines_item_id table\n order_id TEXT REFERENCES orders(order_id), -- Foreign key to the orders table\n sku TEXT NOT NULL -- The purchased article number\n );\n\n```\n\n### Example Object\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `csv`, `json`, `yaml`, `custom` |\n| description | `string` | An optional string describing the example. |\n| model | `string` | The reference to the model in the schema, e.g. a table name. | \n| data | `string` | Example data for this model. |\n\nExample:\n\n```yaml\nexamples:\n- type: csv\n model: orders\n data: |-\n order_id,order_timestamp,order_total\n \"1001\",\"2023-09-09T08:30:00Z\",2500\n \"1002\",\"2023-09-08T15:45:00Z\",1800\n \"1003\",\"2023-09-07T12:15:00Z\",3200\n \"1004\",\"2023-09-06T19:20:00Z\",1500\n \"1005\",\"2023-09-05T10:10:00Z\",4200\n \"1006\",\"2023-09-04T14:55:00Z\",2800\n \"1007\",\"2023-09-03T21:05:00Z\",1900\n \"1008\",\"2023-09-02T17:40:00Z\",3600\n \"1009\",\"2023-09-01T09:25:00Z\",3100\n \"1010\",\"2023-08-31T22:50:00Z\",2700\n```\n\n### Quality Object\n\nThe quality object contains quality attributes and checks.\n\n| Field | Type | Description |\n| ----- |-------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `SodaCL`, `montecarlo`, `custom` |\n| specification | [SodaCL Quality Object](#sodacl-quality-object) \\|
[Monte Carlo Schema Object](#monte-carlo-quality-object) \\|
`string` | REQUIRED. The specification of the quality attributes. The quality specification can be encoded as a string or as inline YAML. |\n\n\n#### SodaCL Quality Object\n\nQuality attributes in [Soda Checks Language](https://docs.soda.io/soda-cl/soda-cl-overview.html).\n\nThe `specification` represents the content of a `checks.yml` file.\n\nExample (inline):\n\n```yaml\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, dbt-tests, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - row_count > 0\n - duplicate_count(order_id) = 0\n checks for line_items:\n - row_count > 0\n```\n\nExample (string):\n\n```yaml\nquality:\n type: SodaCL\n specification: |-\n checks for search_queries:\n - freshness(search_timestamp) < 1d\n - row_count > 100000\n - missing_count(search_query) = 0\n```\n\n#### Monte Carlo Quality Object\n\nQuality attributes defined as Monte Carlos [Monitors as Code](https://docs.getmontecarlo.com/docs/monitors-as-code).\n\nThe `specification` represents the content of a `montecarlo.yml` file.\n\nExample (string):\n\n```yaml\nquality:\n type: montecarlo\n specification: |-\n montecarlo:\n field_health:\n - table: project:dataset.table_name\n timestamp_field: created\n dimension_tracking:\n - table: project:dataset.table_name\n timestamp_field: created\n field: order_status\n```\n\n### Data Types\n\nThe following data types are supported for model fields and definitions:\n\n- Unicode character sequence: `string`, `text`, `varchar`\n- Any numeric type, either integers or floating point numbers: `number`, `decimal`, `numeric`\n- 32-bit signed integer: `int`, `integer`\n- 64-bit signed integer: `long`, `bigint`\n- Single precision (32-bit) IEEE 754 floating-point number: `float`\n- Double precision (64-bit) IEEE 754 floating-point number: `double`\n- Binary value: `boolean`\n- Timestamp with timezone: `timestamp`, `timestamp_tz`\n- Timestamp with no timezone: `timestamp_ntz`\n- Date with no time information: `date`\n- Array: `array`\n- Sequence of 8-bit unsigned bytes: `bytes`\n- Complex type: `object`, `record`, `struct`\n- No value: `null`\n\n### Specification Extensions\n\nWhile the Data Contract Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points.\n\nA custom fields can be added with any name. The value can be null, a primitive, an array or an object.\n\n### Design Principles\n\nThe Data Contract Specification follows these design principles:\n\n- A free, open, and open-sourced standard\n- Follow OpenAPI and AsyncAPI conventions so that it feels immediately familiar\n- Support contract-first approaches\n- Support code-first approaches\n- Support tooling by being machine-readable\n\nTooling\n---\n- [Data Contract CLI](https://github.com/datacontract/cli) is a free CLI tool to help you create, develop, and maintain your data contracts.\n- [Data Contract Studio](https://studio.datacontract.com/) is a free web tool to develop and share data contracts.\n- [Data Mesh Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data products and data contracts. It supports the data contract specification and allows the user to import or export data contracts using this specification.\n\n\nOther Data Contract Specifications\n---\n- [AIDA User Group's Open Data Contract Standard](https://github.com/AIDAUserGroup/open-data-contract-standard)\n- [PayPal's Data Contract Template](https://github.com/paypal/data-contract-template/blob/main/docs/README.md)\n\nLiterature\n---\n- [Driving Data Quality with Data Contracts](https://www.amazon.com/dp/B0C37FPH3D) by Andrew Jones\n\nAuthors\n---\nThe Data Contract Specification was originally created by [Jochen Christ](https://www.linkedin.com/in/jochenchrist/) and [Dr. Simon Harrer](https://www.linkedin.com/in/simonharrer/), and is currently maintained by them.\n\n\nContributing\n---\nContributions are welcome! Please open an issue or a pull request.\n\nLicense\n---\n[MIT License](LICENSE)\n\n\n\n" }, { "path": "versions/0.9.2/datacontract.init.yaml", "content": "dataContractSpecification: 0.9.2\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# production:\n# type: s3\n# location: s3://\n# format: parquet\n# delimiter: new_line\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### models\n\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type:\n# description:\n\n\n### definitions\n\n# definitions:\n# my_field:\n# domain:\n# name:\n# title:\n# type:\n# description:\n# example:\n# pii:\n# classification:\n\n\n### examples\n\n#examples:\n# - type: csv\n# model: my_model\n# data: |-\n# id,timestamp,amount\n# \"1001\",\"2023-09-09T08:30:00Z\",2500\n# \"1002\",\"2023-09-08T15:45:00Z\",1800\n\n\n### quality\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_model: |-\n# - duplicate_count(id) = 0\n" }, { "path": "versions/0.9.2/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"title\": \"DataContractSpecification\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"title\": \"DataContractSpecificationVersion\",\n \"enum\": [\n \"0.9.2\",\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\"\n }\n },\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"additionalProperties\": {\n \"oneOf\": [\n {\n \"type\": \"object\",\n \"title\": \"BigQueryServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"bigquery\",\n \"BigQuery\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"project\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"project\",\n \"dataset\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"S3Server\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"s3\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"An optional string describing the server. Must be in the form of a URL.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"location\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"RedshiftServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"redshift\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"SnowflakeServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"snowflake\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"DatabricksServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"databricks\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Databricks host\",\n \"examples\": [\"dbc-abcdefgh-1234.cloud.databricks.com\"]\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the Hive or Unity catalog\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The schema name in the catalog\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"catalog\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"PostgresServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"postgres\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\"localhost\"]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\"postgres\"]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\"public\"]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"port\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"KafkaServer\",\n \"description\": \"Kafka Server\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"kafka\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The bootstrap server of the kafka cluster.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the message. Examples: json, avro, protobuf. Default: json.\",\n \"default\": \"json\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"topic\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"PubSubServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"pubsub\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"project\",\n \"topic\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"LocalServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"local\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"path\": {\n \"type\": \"string\",\n \"description\": \"The relative or absolute path to the data file(s).\",\n \"examples\": [\n \"./folder/data.parquet\",\n \"./folder/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the file(s)\",\n \"examples\": [\"json\", \"parquet\", \"csv\"]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"path\",\n \"format\"\n ]\n }\n\n ]\n },\n \"description\": \"Information about the servers.\"\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n }\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Model\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, view, object. Default: table.\",\n \"type\": \"string\",\n \"title\": \"ModelType\",\n \"default\": \"table\",\n \"enum\": [\n \"table\",\n \"view\",\n \"object\"\n ]\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Field\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"title\": \"FieldType\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\",\n \"decimal\",\n \"numeric\",\n \"int\",\n \"integer\",\n \"long\",\n \"bigint\",\n \"float\",\n \"double\",\n \"string\",\n \"text\",\n \"varchar\",\n \"boolean\",\n \"timestamp\",\n \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"array\",\n \"object\",\n \"record\",\n \"struct\",\n \"bytes\",\n \"null\"\n ]\n },\n \"required\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if this field must contain a value and may not be null.\"\n },\n \"primary\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"If this field is a primary key.\"\n },\n \"unique\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if the value must be unique within the model.\"\n },\n \"enum\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"uniqueItems\": true,\n \"description\": \"A value must be equal to one of the elements in this array value. Only evaluated if the value is not null.\"\n },\n \"minLength\": {\n \"type\": \"number\",\n \"description\": \"A value must greater than, or equal to, the value of this. Only applies to string types.\"\n },\n \"maxLength\": {\n \"type\": \"number\",\n \"description\": \"A value must less than, or equal to, the value of this. Only applies to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"A specific format the value must comply with (e.g., 'email', 'uri', 'uuid').\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression the value must match. Only applies to string types.\"\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\n \"sensitive\",\n \"restricted\",\n \"internal\",\n \"public\"\n ]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n }\n }\n }\n }\n }\n }\n },\n \"definitions\": {\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"type\": \"object\",\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Definition\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type.\"\n },\n \"minLength\": {\n \"type\": \"number\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"number\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n }\n },\n \"required\": [\n \"name\",\n \"type\"\n ]\n }\n },\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"title\": \"SchemaType\",\n \"enum\": [\n \"dbt\",\n \"bigquery\",\n \"json-schema\",\n \"sql-ddl\",\n \"avro\",\n \"protobuf\",\n \"custom\"\n ],\n \"description\": \"The type of the schema. Typical values are dbt, bigquery, json-schema, sql-ddl, avro, protobuf, custom.\"\n },\n \"specification\": {\n \"oneOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the schema as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the schema as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The schema of the data contract describes the syntax and semantics of provided data sets. It supports different schema types.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"title\": \"ExampleType\",\n \"enum\": [\n \"csv\",\n \"json\",\n \"yaml\",\n \"custom\"\n ],\n \"description\": \"The type of the example data. Well-known types are csv, json, yaml, custom.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the example.\"\n },\n \"model\": {\n \"type\": \"string\",\n \"description\": \"The reference to the model in the schema, e.g., a table name.\"\n },\n \"data\": {\n \"oneOf\": [{\n \"type\": \"string\",\n \"description\": \"Example data for this model.\"\n },{\n \"type\": \"array\",\n \"description\": \"Example data for this model in a structured format. Use this for type json or yaml.\"\n }]\n }\n },\n \"required\": [\n \"type\",\n \"data\"\n ]\n },\n \"description\": \"The Examples Object is an array of Example Objects.\"\n },\n \"quality\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"title\": \"QualityType\",\n \"enum\": [\n \"SodaCL\",\n \"montecarlo\",\n \"custom\"\n ],\n \"description\": \"The type of the quality check. Typical values are SodaCL, montecarlo, custom.\"\n },\n \"specification\": {\n \"oneOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the quality attributes as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the quality attributes as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The quality object contains quality attributes and checks.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ]\n}\n" }, { "path": "versions/0.9.3/README.md", "content": "# Data Contract Specification\n\n\n \"Stars\"\n\"Slack\n\n![datacontract.png](images/datacontract.png)\n\nData contracts bring data providers and data consumers together.\n\nA _data contract_ is a document that defines the structure, format, semantics, quality, and terms of use for exchanging data between a data provider and their consumers.\nThink of an API, but for data.\nA data contract is implemented by a data product or other data technologies, even legacy data warehouses.\nData contracts can also be used for the input port to specify the expectations of data dependencies and verify given guarantees.\n\nThe _data contract specification_ defines a YAML format to describe attributes of provided data sets.\nIt is data platform neutral and can be used with any data platform, such as AWS S3, Google BigQuery, Azure, Databricks, and Snowflake.\nThe data contract specification is an open initiative to define a common data contract format.\nIt follows [OpenAPI](https://www.openapis.org/) and [AsyncAPI](https://www.asyncapi.com/) conventions.\n\nData contracts come into play when data is exchanged between different teams or organizational units, such as in a [data mesh architecture](https://www.datamesh-architecture.com/).\nFirst, and foremost, data contracts are a communication tool to express a common understanding of how data should be structured and interpreted.\nThey make semantic and quality expectations explicit.\nThey are often created collaboratively in [workshops](./workshop.md) together with data providers and data consumers.\nLater in development and production, they also serve as the basis for code generation, testing, schema validations, quality checks, monitoring, access control, and computational governance policies.\n\nThe specification comes along with the [Data Contract CLI](https://github.com/datacontract/datacontract-cli), an open-source tool to develop, validate, and enforce data contracts.\n\n> _Note: The term \"data contract\" refers to a specification that is usually owned by the data provider and thus does not align with a \"contract\" in a legal sense as a mutual agreement between two parties.\n> The term \"contract\" may be somewhat misleading, but it is how it is used by the industry.\n> The mutual agreement between one data provider and one data consumer is the \"data usage agreement\" that refers to a data contract.\n> Data usage agreements have a defined lifecycle, start/end date, and help the data provider to track who accesses their data and for which purposes._\n\nVersion\n---\n\n0.9.3 ([Changelog](CHANGELOG.md))\n\nExample\n---\n\n[![Data Contract Catalog](https://img.shields.io/badge/Data%20Contract-Catalog-blue)](https://datacontract.com/examples/index.html)\n\n```yaml\ndataContractSpecification: 0.9.3\nid: urn:datacontract:checkout:orders-latest\ninfo:\n title: Orders Latest\n version: 1.0.0\n description: |\n Successful customer orders in the webshop. \n All orders since 2020-01-01. \n Orders with their line items are in their current state (no history included).\n owner: Checkout Team\n slackChannel: \"#checkout\"\n contact:\n name: John Doe (Data Product Owner)\n url: https://teams.microsoft.com/l/channel/example/checkout\ntags:\n - checkout\n - orders\n - s3\nlinks:\n datacontractCli: https://cli.datacontract.com\nservers:\n production:\n type: s3\n environment: prod\n location: s3://datacontract-example-orders-latest/data/{model}/*.json\n format: json\n delimiter: new_line\n description: \"One folder per model. One file per day.\"\nterms:\n usage: |\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: |\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table\n fields:\n order_id:\n $ref: '#/definitions/order_id'\n required: true\n unique: true\n primary: true\n order_timestamp:\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n type: timestamp\n required: true\n example: \"2024-09-09T08:30:00Z\"\n order_total:\n description: Total amount the smallest monetary unit (e.g., cents).\n type: long\n required: true\n example: \"9999\"\n customer_id:\n description: Unique identifier for the customer.\n type: text\n minLength: 10\n maxLength: 20\n customer_email_address:\n description: The email address, as entered by the customer. The email address was not verified.\n type: text\n format: email\n required: true\n pii: true\n classification: sensitive\n processed_timestamp:\n description: The timestamp when the record was processed by the data platform.\n type: timestamp\n required: true\n config:\n jsonType: string\n jsonFormat: date-time\n line_items:\n description: A single article that is part of an order.\n type: table\n fields:\n lines_item_id:\n type: text\n description: Primary key of the lines_item_id table\n required: true\n unique: true\n primary: true\n order_id:\n $ref: '#/definitions/order_id'\n references: orders.order_id\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\ndefinitions:\n order_id:\n domain: checkout\n name: order_id\n title: Order ID\n type: text\n format: uuid\n description: An internal ID that identifies an order in the online shop.\n example: 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n tags:\n - orders\n sku:\n domain: inventory\n name: sku\n title: Stock Keeping Unit\n type: text\n pattern: ^[A-Za-z0-9]{8,14}$\n example: \"96385074\"\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article. \n It is typically associated with an article's barcode, such as the EAN/GTIN.\n links:\n wikipedia: https://en.wikipedia.org/wiki/Stock_keeping_unit\n tags:\n - inventory\nexamples:\n - type: csv # csv, json, yaml, custom\n model: orders\n description: An example list of order records.\n data: | # expressed as string or inline yaml or via \"$ref: data.csv\"\n order_id,order_timestamp,order_total,customer_id,customer_email_address,processed_timestamp\n \"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\",\"2030-09-09T08:31:00Z\"\n \"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\",\"2030-09-09T08:31:00Z\"\n \"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\",\"2030-09-09T08:31:00Z\"\n \"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\",\"2030-09-09T08:31:00Z\"\n \"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\",\"2030-09-09T08:31:00Z\"\n \"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\",\"2030-09-09T08:31:00Z\"\n \"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\",\"2030-09-09T08:31:00Z\"\n \"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\",\"2030-09-09T08:31:00Z\"\n - type: csv\n model: line_items\n description: An example list of line items.\n data: |\n lines_item_id,order_id,sku\n \"LI-1\",\"1001\",\"5901234123457\"\n \"LI-2\",\"1001\",\"4001234567890\"\n \"LI-3\",\"1002\",\"5901234123457\"\n \"LI-4\",\"1002\",\"2001234567893\"\n \"LI-5\",\"1003\",\"4001234567890\"\n \"LI-6\",\"1003\",\"5001234567892\"\n \"LI-7\",\"1004\",\"5901234123457\"\n \"LI-8\",\"1005\",\"2001234567893\"\n \"LI-9\",\"1005\",\"5001234567892\"\n \"LI-10\",\"1005\",\"6001234567891\"\nservicelevels:\n availability:\n description: The server is available during support hours\n percentage: 99.9%\n retention:\n description: Data is retained for one year\n period: P1Y\n unlimited: false\n latency:\n description: Data is available within 25 hours after the order was placed\n threshold: 25h\n sourceTimestampField: orders.order_timestamp\n processedTimestampField: orders.processed_timestamp\n freshness:\n description: The age of the youngest row in a table.\n threshold: 25h\n timestampField: orders.order_timestamp\n frequency:\n description: Data is delivered once a day\n type: batch # or streaming\n interval: daily # for batch, either or cron\n cron: 0 0 * * * # for batch, either or interval\n support:\n description: The data is available during typical business hours at headquarters\n time: 9am to 5pm in EST on business days\n responseTime: 1h\n backup:\n description: Data is backed up once a week, every Sunday at 0:00 UTC.\n interval: weekly\n cron: 0 0 * * 0\n recoveryTime: 24 hours\n recoveryPoint: 1 week\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - row_count >= 5\n - duplicate_count(order_id) = 0\n checks for line_items:\n - values in (order_id) must exist in orders (order_id)\n - row_count >= 5\n```\n\nData Contract CLI\n---\n\nThe [Data Contract CLI](https://cli.datacontract.com) is a command line tool and Python library to lint, test, import and export data contracts.\n\nHere is short example how to verify that your actual dataset matches the data contract:\n\n```bash\npip3 install datacontract-cli\ndatacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nor, if you prefer Docker:\n```bash\ndocker run datacontract/cli test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nThe Data Contract contains all required information to verify data:\n\n- The _servers_ block has the connection details to the actual data set.\n- The _models_ define the syntax, formats, and constraints.\n- The _quality_ defined further quality checks.\n\nThe Data Contract CLI chooses the appropriate engine, formulates test cases, connects to the server, and executes the tests, based on the server type.\n\nMore information and configuration options on [cli.datacontract.com](https://cli.datacontract.com).\n\n\nSpecification\n---\n\n![The eight major categories in the data contract specification](images/categories.png)\n\n- [Data Contract Object](#data-contract-object)\n- [Info Object](#info-object)\n- [Contact Object](#contact-object)\n- [Server Object](#server-object)\n- [Terms Object](#terms-object)\n- [Model Object](#model-object)\n- [Field Object](#field-object)\n- [Definition Object](#definition-object)\n- [Schema Object (DEPRECATED)](#schema-object-deprecated)\n- [Example Object](#example-object)\n- [Service Level Object](#service-levels-object)\n- [Quality Object](#quality-object)\n- [Data Types](#data-types)\n- [Specification Extensions](#specification-extensions)\n\n\n[JSON Schema](https://github.com/datacontract/datacontract-specification/blob/main/datacontract.schema.json) of the Data Contract Specification.\n\n### Data Contract Object\n\nThis is the root document.\n\nIt is _RECOMMENDED_ that the root document be named: `datacontract.yaml`.\n\n| Field | Type | Description |\n|---------------------------|------------------------------------------------------|----------------------------------------------------------------------------------------------------------|\n| dataContractSpecification | `string` | REQUIRED. Specifies the Data Contract Specification being used. |\n| id | `string` | REQUIRED. An organization-wide unique technical identifier, such as a UUID, URN, slug, string, or number |\n| info | [Info Object](#info-object) | REQUIRED. Specifies the metadata of the data contract. May be used by tooling. |\n| servers | Map[`string`, [Server Object](#server-object)] | Specifies the servers of the data contract. |\n| terms | [Terms Object](#terms-object) | Specifies the terms and conditions of the data contract. |\n| models | Map[`string`, [Model Object](#model-object)] | Specifies the logical data model. |\n| definitions | Map[`string`, [Definition Object](#definition-object)] | Specifies definitions. |\n| schema | [Schema Object (DEPRECATED)](#schema-object-deprecated) | Specifies the physical schema. The specification supports different schema format. |\n| examples | Array of [Example Objects](#example-object) | Specifies example data sets for the data model. The specification supports different example types. |\n| servicelevels | [Service Levels Object](#service-levels-object) | Specifies the service level of the provided data |\n| quality | [Quality Object](#quality-object) | Specifies the quality attributes and checks. The specification supports different quality check DSLs. |\n| links | Map[`string`, `string`] | Additional external documentation links. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n\n### Info Object\n\nMetadata and life cycle information about the data contract.\n\n\n| Field | Type | Description |\n|-------------|-----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| title | `string` | REQUIRED. The title of the data contract. |\n| version | `string` | REQUIRED. The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version). |\n| status | `string` | The status of the data contract. Can be `proposed`, `in development`, `active`, `deprecated`, `retired`. |\n| description | `string` | A description of the data contract. |\n| owner | `string` | The owner or team responsible for managing the data contract and providing the data. |\n| contact | [Contact Object](#contact-object) | Contact information for the data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Contact Object\n\nContact information for the data contract.\n\n| Field | Type | Description |\n|-------|----------|-------------------------------------------------------------------------------------------------------|\n| name | `string` | The identifying name of the contact person/organization. |\n| url | `string` | The URL pointing to the contact information. This _MUST_ be in the form of a URL. |\n| email | `string` | The email address of the contact person/organization. This _MUST_ be in the form of an email address. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n### Server Object\n\nThe fields are dependent on the defined type.\n\n| Field | Type | Description |\n|-------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `s3`, `glue`, `redshift`, `azure`, `sqlserver`, `snowflake`, `databricks`, `postgres`, `oracle`, `kafka`, `pubsub`, `sftp`, `kinesis`, `trino`, `local` |\n| description | `string` | An optional string describing the server. |\n| environment | `string` | An optional string describing the environment, e.g., prod, sit, stg. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### BigQuery Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `bigquery` |\n| project | `string` | The GCP project name. |\n| dataset | `string` | |\n\n#### S3 Server Object\n\n| Field | Type | Description |\n|-------------|----------|-------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `s3` |\n| location | `string` | S3 URL, starting with `s3://` |\n| endpointUrl | `string` | The server endpoint for S3-compatible servers, such as MioIO or Google Cloud Storage, e.g., `https://minio.example.com` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\nExample (AWS S3):\n\n```yaml\nservers:\n production:\n type: s3\n location: s3://acme-orders-prod/orders/\n format: json\n delimiter: new_line\n```\n\nExample (MinIO):\n\n```yaml\nservers:\n minio:\n type: s3\n endpointUrl: http://localhost:9000\n location: s3://my-bucket/path/\n format: delta\n```\n\nExample (Google Cloud Storage):\n\n```yaml\nservers:\n gcs:\n type: s3\n endpointUrl: https://storage.googleapis.com\n location: s3://my-bucket/path/*/*/*/*/*.parquet\n format: parquet\n```\n\n\n#### Redshift Server Object\n\n| Field | Type | Description |\n|-------------------|----------|---------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `redshift` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n| clusterIdentifier | `string` | Identifier of the cluster.
Example: `analytics-cluster` |\n| host | `string` | Host of the cluster.
Example: `analytics-cluster.example.eu-west-1.redshift.amazonaws.com` |\n| port | `number` | Port of the cluster.
Example: `5439` |\n| endpoint | `string` | Endpoint of the cluster
Example: `analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics` |\n\nExample, specifying an endpoint:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n endpoint: analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\n```\n\nExample, specifying the cluster identifier:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n clusterIdentifier: analytics-cluster\n```\n\nExample, specifying the cluster host:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n host: analytics-cluster.example.eu-west-1.redshift.amazonaws.com\n port: 5439\n```\n\n#### Azure Server Object\n\n| Field | Type | Description |\n|-----------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `azure` |\n| location | `string` | Fully qualified path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs. Starting with `az://` or `abfss`
Examples: `az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet` or `abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet` |\n| format | `string` | Format of files, such as `parquet`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\n\n#### SQL-Server Server Object\n\n| Field | Type | Description |\n|----------|-----------|------------------------------------------------------|\n| type | `string` | `sqlserver` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server, default: `1433` |\n| database | `string` | The name of the database, e.g., `database`. |\n| schema | `string` | The name of the schema in the database, e.g., `dbo`. |\n| driver | `string` | The name of the supported driver, e.g., `ODBC Driver 18 for SQL Server`. |\n\n\n#### Snowflake Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `snowflake` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Databricks Server Object\n\n| Field | Type | Description |\n|---------|----------|---------------------------------------------------------------------|\n| type | `string` | `databricks` |\n| host | `string` | The Databricks host, e.g., `dbc-abcdefgh-1234.cloud.databricks.com` |\n| catalog | `string` | The name of the Hive or Unity catalog |\n| schema | `string` | The schema name in the catalog |\n\n#### Postgres Server Object\n\n| Field | Type | Description |\n|----------|-----------|---------------------------------------------------------|\n| type | `string` | `postgres` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server |\n| database | `string` | The name of the database, e.g., `postgres`. |\n| schema | `string` | The name of the schema in the database, e.g., `public`. |\n\n#### Oracle Server Object\n\n| Field | Type | Description |\n|-------------|-----------|---------------------------------|\n| type | `string` | `oracle` |\n| host | `string` | The host to the oracle server |\n| port | `integer` | The port to the oracle server |\n| serviceName | `string` | The name of the service |\n\n#### Kafka Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kafka` |\n| host | `string` | The bootstrap server of the kafka cluster. |\n| topic | `string` | The topic name. |\n| format | `string` | The format of the message. Examples: json, avro, protobuf. Default: json. |\n\n#### Pub/Sub Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `pubsub` |\n| project | `string` | The GCP project name. |\n| topic | `string` | The topic name. |\n\n#### sftp Server Object\n\n| Field | Type | Description |\n|-----------|----------|------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `sftp` |\n| location | `string` | S3 URL, starting with `sftp://` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\n#### AWS Kinesis Data Streams Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kinesis` |\n| stream | `string` | The name of the Kinesis data stream. |\n| region | `string` | AWS region, e.g., `eu-west-1`. |\n| format | `string` | The format of the records. Examples: json, avro, protobuf. |\n\n#### Trino Server Object\n\n| Field | Type | Description |\n|----------|-----------|-----------------------------------------------------------|\n| type | `string` | `trino` |\n| host | `string` | The Trino host |\n| port | `integer` | The Trino port | \n| catalog | `string` | The name of the catalog, e.g., `my_catalog`. |\n| schema | `string` | The name of the schema in the catalog, e.g., `my_schema`. |\n\n#### Local Server Object\n\n| Field | Type | Description |\n|--------|----------|-------------------------------------------------------------------------------------|\n| type | `string` | `local` |\n| path | `string` | The relative or absolute path to the data file(s), such as `./folder/data.parquet`. |\n| format | `string` | The format of the file(s), such as `parquet`, `delta`, `csv`, or `json`. |\n\n### Terms Object\n\nThe terms and conditions of the data contract.\n\n| Field | Type | Description |\n|--------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| usage | `string` | The usage describes the way the data is expected to be used. Can contain business and technical information. |\n| limitations | `string` | The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for. |\n| billing | `string` | The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use. |\n| noticePeriod | `string` | The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., `P3M` for a period of three months. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Model Object\n\nThe Model Object describes the structure and semantics of a data model, such as tables, views, or structured files.\n\nThe name of the data model (table name) is defined by the key that refers to this Model Object.\n\n| Field | Type | Description |\n|-------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the model. Examples: `table`, `view`, `object`. Default: `table`. |\n| description | `string` | An optional string describing the data model. |\n| title | `string` | An optional string for the title of the data model. Especially useful if the name of the model is cryptic or contains abbreviations. |\n| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |\n| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Field Object\n\nThe Field Objects describes one field (column, property, nested field) of a data model.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the semantic of the data in this field. |\n| type | [Data Type](#data-types) | The logical data type of the field. |\n| title | `string` | An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations. |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| required | `boolean` | An indication, if this field must contain a value and may not be null. Default: `false` |\n| primary | `boolean` | If this field is a primary key. Default: `false` |\n| references | `string` | The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship. |\n| unique | `boolean` | An indication, if the value must be unique within the model. Default: `false` |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| precision | `number` | The maximum number of digits in a number. Only applies to numeric values. Defaults to 38. |\n| scale | `number` | The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0. |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| example | `string` | An example value. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). | \n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. Examples may be: `sensitive`, `restricted`, `internal`, `public`. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| links | Map[`string`,`string`] | Additional external documentation links. |\n| $ref | `string` | A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition. |\n| fields | Map[`string`, [Field Object](#field-object)] | The nested fields (e.g. columns) of the object, record, or struct. Use only when type is `object`, `record`, or `struct`. |\n| items | [Field Object](#field-object) | The type of the elements in the array. Use only when type is `array`. |\n| keys | [Field Object](#field-object) | Describes the key structure of a map. Defaults to `type: string` if a map is defined as type. Not all server types support different key types. Use only when type is `map`. |\n| values | [Field Object](#field-object) | Describes the value structure of a map. Use only when type is `map`. |\n| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Definition Object\n\nThe Definition Object includes a clear and concise explanations of syntax, semantic, and classification of a business object in a given domain.\nIt serves as a reference for a common understanding of terminology, ensure consistent usage and to identify join-able fields.\nModels fields can refer to definitions using the `$ref` field to link to existing definitions and avoid duplicate documentations.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| name | `string` | REQUIRED. The technical name of this definition. |\n| type | [Data Type](#data-types) | REQUIRED. The logical data type |\n| domain | `string` | The domain in which this definition is valid. Default: `global`. |\n| title | `string` | The business name of this definition. |\n| description | `string` | Clear and concise explanations related to the domain |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| precision | `number` | The maximum number of digits in a number. Only applies to numeric values. Defaults to 38. |\n| scale | `number` | The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0. |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| example | `string` | An example value. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). |\n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| links | Map[`string`, `string`] | Additional external documentation links. |\n| fields | Map[`string`, [Field Object](#field-object)] | The nested fields (e.g. columns) of the object, record, or struct. Use only when type is `object`, `record`, or `struct`. |\n| items | [Field Object](#field-object) | The type of the elements in the array. Use only when type is `array`. |\n| keys | [Field Object](#field-object) | Describes the key structure of a map. Defaults to `type: string` if a map is defined as type. Not all server types support different key types. Use only when type is `map`. |\n| values | [Field Object](#field-object) | Describes the value structure of a map. Use only when type is `map`. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Schema Object (DEPRECATED)\n\nThe schema of the data contract describes the physical schema.\nThe type of the schema depends on the data platform.\n\n| Field | Type | Description |\n|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `dbt`, `bigquery`, `json-schema`, `sql-ddl`, `avro`, `protobuf`, `custom` |\n| specification | [dbt Schema Object](#dbt-schema-object) \\|
[BigQuery Schema Object](#bigquery-schema-object) \\|
[JSON Schema Schema Object](#bigquery-schema-object) \\|
[SQL DDL Schema Object](#sql-ddl-schema-object) \\|
`string` | REQUIRED. The specification of the schema. The schema specification can be encoded as a string or as inline YAML. |\n\n\n#### dbt Schema Object\n\nhttps://docs.getdbt.com/reference/model-properties\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: dbt\n specification:\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\nExample (string):\n\n```yaml\nschema:\n type: dbt\n specification: |-\n version: 2\n models:\n - name: \"My Table\"\n description: \"My description\"\n columns:\n - name: \"My column\"\n data_type: text\n description: \"My description\"\n```\n\n#### BigQuery Schema Object\n\nThe schema structure is defined by the [Google BigQuery Table](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#resource:-table) object. You can extract such a Table object via the [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) endpoint.\n\nInstead of providing a single Table object, you can also provide an array of such objects. Be aware that [tables.list](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list) only returns a subset of the full Table object. You need to call every Table object via [tables.get](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/get) to get the full Table object, including the actual schema.\n\nLearn more: [Google BigQuery REST Reference v2](https://cloud.google.com/bigquery/docs/reference/rest)\n\n\n\nExample:\n\n```yaml\nschema:\n type: bigquery\n specification: |-\n {\n \"tableReference\": {\n \"projectId\": \"my-project\",\n \"datasetId\": \"my_dataset\",\n \"tableId\": \"my_table\"\n },\n \"description\": \"This is a description\",\n \"type\": \"TABLE\",\n \"schema\": {\n \"fields\": [\n {\n \"name\": \"name\",\n \"type\": \"STRING\",\n \"mode\": \"NULLABLE\",\n \"description\": \"This is a description\"\n }\n ]\n }\n }\n```\n\n#### JSON Schema Schema Object\n\nJSON Schema can be defined as JSON or rendered as YAML, following the [OpenAPI Schema Object dialect](https://spec.openapis.org/oas/v3.1.0#properties)\n\nExample (inline YAML):\n\n```yaml\nschema:\n type: json-schema\n specification:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: object\n properties:\n order_id:\n type: string\n description: Primary key of the orders table\n order_timestamp:\n type: string\n format: date-time\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total:\n type: integer\n description: Total amount of the order in the smallest monetary unit (e.g., cents).\n line_items:\n type: object\n properties:\n lines_item_id:\n type: string\n description: Primary key of the lines_item_id table\n order_id:\n type: string\n description: Foreign key to the orders table\n sku:\n type: string\n description: The purchased article number\n```\n\nExample (string):\n\n```yaml\nschema:\n type: json-schema\n specification: |-\n {\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"properties\": {\n \"orders\": {\n \"type\": \"object\",\n \"description\": \"One record per order. Includes cancelled and deleted orders.\",\n \"properties\": {\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the orders table\"\n },\n \"order_timestamp\": {\n \"type\": \"string\",\n \"format\": \"date-time\",\n \"description\": \"The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\"\n },\n \"order_total\": {\n \"type\": \"integer\",\n \"description\": \"Total amount of the order in the smallest monetary unit (e.g., cents).\"\n }\n },\n \"required\": [\"order_id\", \"order_timestamp\", \"order_total\"]\n },\n \"line_items\": {\n \"type\": \"object\",\n \"properties\": {\n \"lines_item_id\": {\n \"type\": \"string\",\n \"description\": \"Primary key of the lines_item_id table\"\n },\n \"order_id\": {\n \"type\": \"string\",\n \"description\": \"Foreign key to the orders table\"\n },\n \"sku\": {\n \"type\": \"string\",\n \"description\": \"The purchased article number\"\n }\n },\n \"required\": [\"lines_item_id\", \"order_id\", \"sku\"]\n }\n },\n \"required\": [\"orders\", \"line_items\"]\n }\n```\n\n#### SQL DDL Schema Object\n\nClassical SQL DDLs can be used to describe the structure.\n\n\nExample (string):\n\n```yaml\nschema:\n type: sql-ddl\n specification: |-\n -- One record per order. Includes cancelled and deleted orders.\n CREATE TABLE orders (\n order_id TEXT PRIMARY KEY, -- Primary key of the orders table\n order_timestamp TIMESTAMPTZ NOT NULL, -- The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n order_total INTEGER NOT NULL -- Total amount of the order in the smallest monetary unit (e.g., cents)\n );\n \n -- The items that are part of an order\n CREATE TABLE line_items (\n lines_item_id TEXT PRIMARY KEY, -- Primary key of the lines_item_id table\n order_id TEXT REFERENCES orders(order_id), -- Foreign key to the orders table\n sku TEXT NOT NULL -- The purchased article number\n );\n\n```\n\n### Example Object\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the data product technology that implements the data contract. Well-known server types are: `csv`, `json`, `yaml`, `custom` |\n| description | `string` | An optional string describing the example. |\n| model | `string` | The reference to the model in the schema, e.g. a table name. | \n| data | `string` | Example data for this model. |\n\nExample:\n\n```yaml\nexamples:\n- type: csv\n model: orders\n data: |-\n order_id,order_timestamp,order_total\n \"1001\",\"2023-09-09T08:30:00Z\",2500\n \"1002\",\"2023-09-08T15:45:00Z\",1800\n \"1003\",\"2023-09-07T12:15:00Z\",3200\n \"1004\",\"2023-09-06T19:20:00Z\",1500\n \"1005\",\"2023-09-05T10:10:00Z\",4200\n \"1006\",\"2023-09-04T14:55:00Z\",2800\n \"1007\",\"2023-09-03T21:05:00Z\",1900\n \"1008\",\"2023-09-02T17:40:00Z\",3600\n \"1009\",\"2023-09-01T09:25:00Z\",3100\n \"1010\",\"2023-08-31T22:50:00Z\",2700\n```\n\n### Service Levels Object\n\nA service level is defined as an agreed-upon, measurable level of performance for provided the data.\nData Contract Specification defines well-known service levels.\nThis list can be extended with custom service levels.\n\nOne can either describe each service level informally using the `description` field, or make use of the predefined fields for automation support, e.g., via the [Data Contract CLI](https://cli.datacontract.com).\n\n| Field | Type | Description |\n|--------------|-----------------------------------------------|-------------------------------------------------------------------------|\n| availability | [Availability Object](#availability-object) | The promised uptime of the system that provides the data |\n| retention | [Retention Object](#retention-object) | The period how long data will be available. |\n| latency | [Latency Object](#latency-object) | The maximum amount of time from the source to its destination. |\n| freshness | [Freshness Object](#freshness-object) | The maximum age of the youngest entry. |\n| frequency | [Frequency Object](#frequency-object) | The update frequency. |\n| support | [Support Object](#support-object) | The times when support is provided. |\n| backup | [Backup Object](#backup-object) | The details about data backup procedures. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Availability Object\n\nAvailability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the availability service level. |\n| percentage | `string` | An optional string describing the guaranteed uptime in percent (e.g., `99.9%`) |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Retention Object\n\nRetention covers the period how long data will be available.\n\n| Field | Type | Description |\n|----------------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the retention service level. |\n| period | `string` | An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`). |\n| unlimited | `boolean` | An optional indicator that data is kept forever. |\n| timestampField | `string` | An optional reference to the field that contains the timestamp that the period refers to. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Latency Object\n\nLatency refers to the maximum amount of time from the source to its destination.\n\nExamples are the maximum duration it takes after an order has been recorded in the ecommerce shop until it is available in the orders table in the data analytics platform. This includes the waiting times until the next batch run is started and the processing time of the pipeline.\n\n| Field | Type | Description |\n|-------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the latency service level. |\n| threshold | `string` | An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`). |\n| sourceTimestampField | `string` | An optional reference to the field that contains the timestamp when the data was provided at the source. |\n| processedTimestampField | `string` | An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Freshness Object\n\nFreshness refers to the maximum age of the youngest entry.\n\n| Field | Type | Description |\n|-------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the freshness service level. |\n| threshold | `string` | An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`). |\n| timestampField | `string` | An optional reference to the field that contains the timestamp that the threshold refers to. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Frequency Object\n\nFrequency describes how often data is updated.\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the frequency service level. |\n| type | `string` | An optional type of data processing. Typical values are `batch`, `micro-batching`, `streaming`, `manual`. |\n| interval | `string` | Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`. |\n| cron | `string` | Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n#### Support Object\n\nSupport describes the times when support will be available for contact.\n\n| Field | Type | Description |\n|--------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the support service level. |\n| time | `string` | An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`. |\n| responseTime | `string` | An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with. |\n\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n#### Backup Object\n\nBackup specifies details about data backup procedures.\n\n| Field | Type | Description |\n|---------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the backup service level. |\n| interval | `string` | An optional interval that defines how often data will be backed up, e.g., `daily`. |\n| cron | `string` | An optional cron expression when data will be backed up, e.g., `0 0 * * *`. |\n| recoveryTime | `string` | An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours). |\n| recoveryPoint | `string` | An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours). |\n\n\n\n### Quality Object\n\nThe quality object contains quality attributes and checks.\n\n| Field | Type | Description |\n|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the schema.
Typical values are: `SodaCL`, `montecarlo`, `great-expectations`, `custom` |\n| specification | [SodaCL Quality Object](#sodacl-quality-object) \\|
[Monte Carlo Schema Object](#monte-carlo-quality-object) \\|
[Great Expectations Quality Object](#great-expectations-quality-object) \\|
`string` | REQUIRED. The specification of the quality attributes. The quality specification can be encoded as a string or as inline YAML. |\n\n\n#### SodaCL Quality Object\n\nQuality attributes in [Soda Checks Language](https://docs.soda.io/soda-cl/soda-cl-overview.html).\n\nThe `specification` represents the content of a `checks.yml` file.\n\nExample (inline):\n\n```yaml\nquality:\n type: SodaCL # data quality check format: SodaCL, montecarlo, dbt-tests, custom\n specification: # expressed as string or inline yaml or via \"$ref: checks.yaml\"\n checks for orders:\n - row_count > 0\n - duplicate_count(order_id) = 0\n checks for line_items:\n - row_count > 0\n```\n\nExample (string):\n\n```yaml\nquality:\n type: SodaCL\n specification: |-\n checks for search_queries:\n - freshness(search_timestamp) < 1d\n - row_count > 100000\n - missing_count(search_query) = 0\n```\n\n#### Monte Carlo Quality Object\n\nQuality attributes defined as Monte Carlos [Monitors as Code](https://docs.getmontecarlo.com/docs/monitors-as-code).\n\nThe `specification` represents the content of a `montecarlo.yml` file.\n\nExample (string):\n\n```yaml\nquality:\n type: montecarlo\n specification: |-\n montecarlo:\n field_health:\n - table: project:dataset.table_name\n timestamp_field: created\n dimension_tracking:\n - table: project:dataset.table_name\n timestamp_field: created\n field: order_status\n```\n\n#### Great Expectations Quality Object\n\nQuality attributes defined as Great Expectations [Expectations](https://greatexpectations.io/expectations/).\n\nThe `specification` represents a list of expectations on a specific model.\n\nExample (string):\n\n```yaml\nquality:\n type: great-expectations\n specification:\n orders: |-\n [\n {\n \"expectation_type\": \"expect_table_row_count_to_be_between\",\n \"kwargs\": {\n \"min_value\": 10\n },\n \"meta\": {\n \n }\n }\n ]\n```\n\n### Config Object\n\nThe config field can be used to set additional metadata that may be used by tools, e.g. to define a namespace for code generation, specify physical data types, toggle tests, etc.\n\nA config field can be added with any name. The value can be null, a primitive, an array or an object.\n\nFor developer experience, a list of well-known field names is maintained here, as these fields are used in the Data Contract CLI:\n\n\n| Field | Type | Description |\n|-----------------|----------|----------------------------------------------------------------------------------------------------------------|\n| avroNamespace | `string` | (Only on model level) The namespace to use when importing and exporting the data model from / to Apache Avro. |\n| avroType | `string` | (Only on field level) Specify the field type to use when exporting the data model to Apache Avro. |\n| avroLogicalType | `string` | (Only on field level) Specify the logical field type to use when exporting the data model to Apache Avro. |\n| bigqueryType | `string` | (Only on field level) Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)` |\n| snowflakeType | `string` | (Only on field level) Specify the physical column type that is used in a Snowflake table, e.g, `TIMESTAMP_LTZ` |\n| redshiftType | `string` | (Only on field level) Specify the physical column type that is used in a Redshift table, e.g, `SMALLINT` |\n| sqlserverType | `string` | (Only on field level) Specify the physical column type that is used in a Snowflake table, e.g, `DATETIME2` |\n| databricksType | `string` | (Only on field level) Specify the physical column type that is used in a Databricks table |\n| glueType | `string` | (Only on field level) Specify the physical column type that is used in a AWS Glue Data Catalog table |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\nExample:\n\n```\nmodels:\n orders:\n config:\n avroNamespace: \"my.namespace\"\n fields:\n my_field_1:\n description: Example for AVRO with Timestamp (millisecond precision)\n type: timestamp\n config:\n avroType: long\n avroLogicalType: timestamp-millis\n snowflakeType: timestamp_tz\n```\n\n\n### Data Types\n\nThe following data types are supported for model fields and definitions:\n\n- Unicode character sequence: `string`, `text`, `varchar`\n- Any numeric type, either integers or floating point numbers: `number`, `decimal`, `numeric`\n- 32-bit signed integer: `int`, `integer`\n- 64-bit signed integer: `long`, `bigint`\n- Single precision (32-bit) IEEE 754 floating-point number: `float`\n- Double precision (64-bit) IEEE 754 floating-point number: `double`\n- Binary value: `boolean`\n- Timestamp with timezone: `timestamp`, `timestamp_tz`\n- Timestamp with no timezone: `timestamp_ntz`\n- Date with no time information: `date`\n- Array: `array`\n- Map: `map` (may not be supported by some server types)\n- Sequence of 8-bit unsigned bytes: `bytes`\n- Complex type: `object`, `record`, `struct`\n- No value: `null`\n\n### Specification Extensions\n\nWhile the Data Contract Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points.\n\nA custom field can be added with any name. The value can be null, a primitive, an array or an object.\n\n\nTooling\n---\n- [Data Contract CLI](https://github.com/datacontract/datacontract-cli) is an open-source CLI tool to help you create, develop, and maintain your data contracts.\n- [Data Contract Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data contracts. It includes a data contract catalog, a Web-Editor, and a request and approval workflow to automate access to data products for a full enterprise data marketplace.\n- [Data Contract GPT](https://gpt.datacontract.com) is a custom GPT that can help you write data contracts.\n- [Data Contract Editor](https://editor.datacontract.com) is an open-source editor for Data Contracts, including a live html preview.\n\nCode Completion\n---\nThe [JSON Schema](https://datacontract.com/datacontract.schema.json) of the current data contract specification is registered in [Schema Store](https://www.schemastore.org/), which brings code completion and syntax checks for all major IDEs.\nIntelliJ comes with a built-in YAML plugin which will show you autocompletions.\nFor VS Code we recommend to install the [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) plugin.\nNo additional configuration is required.\n\nAutocompletion is then enabled for files following these patterns:\n\n```\ndatacontract.yaml\ndatacontract.yml\n*-datacontract.yaml\n*-datacontract.yml\n*.datacontract.yaml\n*.datacontract.yml\ndatacontract-*.yaml\ndatacontract-*.yml\n**/datacontract/*.yml\n**/datacontract/*.yaml\n**/datacontracts/*.yml\n**/datacontracts/*.yaml\n```\n\n\nAuthors\n---\nThe Data Contract Specification was originally created by [Jochen Christ](https://www.linkedin.com/in/jochenchrist/) and [Dr. Simon Harrer](https://www.linkedin.com/in/simonharrer/), and is currently maintained by them.\n\n\nContributing\n---\nContributions are welcome! Please open an issue or a pull request.\n\nLicense\n---\n[MIT License](LICENSE)\n\n\n" }, { "path": "versions/0.9.3/datacontract.init.yaml", "content": "dataContractSpecification: 0.9.3\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# production:\n# type: s3\n# location: s3://\n# format: parquet\n# delimiter: new_line\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### models\n\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type:\n# description:\n\n\n### definitions\n\n# definitions:\n# my_field:\n# domain:\n# name:\n# title:\n# type:\n# description:\n# example:\n# pii:\n# classification:\n\n\n### examples\n\n#examples:\n# - type: csv\n# model: my_model\n# data: |-\n# id,timestamp,amount\n# \"1001\",\"2023-09-09T08:30:00Z\",2500\n# \"1002\",\"2023-09-08T15:45:00Z\",1800\n\n### servicelevels\n\n#servicelevels:\n# availability:\n# description: The server is available during support hours\n# percentage: 99.9%\n# retention:\n# description: Data is retained for one year because!\n# period: P1Y\n# unlimited: false\n# latency:\n# description: Data is available within 25 hours after the order was placed\n# threshold: 25h\n# sourceTimestampField: orders.order_timestamp\n# processedTimestampField: orders.processed_timestamp\n# freshness:\n# description: The age of the youngest row in a table.\n# threshold: 25h\n# timestampField: orders.order_timestamp\n# frequency:\n# description: Data is delivered once a day\n# type: batch # or streaming\n# interval: daily # for batch, either or cron\n# cron: 0 0 * * * # for batch, either or interval\n# support:\n# description: The data is available during typical business hours at headquarters\n# time: 9am to 5pm in EST on business days\n# responseTime: 1h\n# backup:\n# description: Data is backed up once a week, every Sunday at 0:00 UTC.\n# interval: weekly\n# cron: 0 0 * * 0\n# recoveryTime: 24 hours\n# recoveryPoint: 1 week\n\n### quality\n\n#quality:\n# type: SodaCL\n# specification:\n# checks for my_model: |-\n# - duplicate_count(id) = 0" }, { "path": "versions/0.9.3/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"title\": \"DataContractSpecification\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"title\": \"DataContractSpecificationVersion\",\n \"enum\": [\n \"0.9.3\",\n \"0.9.2\",\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"status\": {\n \"type\": \"string\",\n \"description\": \"The status of the data contract. Can be proposed, in development, active, retired.\",\n \"x-extensible-enum\": [\n \"proposed\",\n \"in development\",\n \"active\",\n \"deprecated\",\n \"retired\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\",\n \"additionalProperties\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the servers.\"\n },\n \"environment\": {\n \"type\": \"string\",\n \"description\": \"The environment in which the servers are running. Examples: prod, sit, stg.\"\n }\n },\n \"additionalProperties\": {\n \"oneOf\": [\n {\n \"type\": \"object\",\n \"title\": \"BigQueryServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"bigquery\",\n \"BigQuery\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"project\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"project\",\n \"dataset\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"S3Server\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"s3\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"An optional string describing the server. Must be in the form of a URL.\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}/*.json\"\n ]\n },\n \"endpointUrl\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The server endpoint for S3-compatible servers.\",\n \"examples\": [\"https://minio.example.com\"]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"location\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"GcsServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"gcs\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The GS/GCS url to the data.\",\n \"examples\": [\n \"gs://example-storage/data/*/*.json\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"location\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"SftpServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"sftp\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"An optional string describing the server. Must be in the form of a sftp URL.\",\n \"examples\": [\n \"sftp://123.123.12.123/{model}/*.json\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"location\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"RedshiftServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"redshift\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the host name.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"clusterIdentifier\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's identifier.\",\n \"examples\": [\n \"redshift-prod-eu\",\n \"analytics-cluster\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"An optional string describing the cluster's port.\",\n \"examples\": [\n 5439\n ]\n },\n \"endpoint\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's endpoint.\",\n \"examples\": [\n \"analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"AzureServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"azure\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"Fully qualified path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs.\",\n \"examples\": [\n \"az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet\",\n \"abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"location\",\n \"format\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"SqlserverServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"sqlserver\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\",\n \"default\": 1433,\n \"examples\": [\n 1433\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"database\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"dbo\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"SnowflakeServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"snowflake\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"DatabricksServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"databricks\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Databricks host\",\n \"examples\": [\n \"dbc-abcdefgh-1234.cloud.databricks.com\"\n ]\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the Hive or Unity catalog\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The schema name in the catalog\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"catalog\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"DataframeServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"dataframe\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"GlueServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"glue\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"account\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue account\",\n \"examples\": [\n \"1234-5678-9012\"\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue database name\",\n \"examples\": [\n \"my_database\"\n ]\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The AWS S3 path. Must be in the form of a URL.\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the files\",\n \"examples\": [\n \"parquet\",\n \"csv\",\n \"json\",\n \"delta\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"account\",\n \"database\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"PostgresServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"postgres\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"public\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"port\",\n \"database\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"OracleServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"oracle\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the oracle server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the oracle server.\",\n \"examples\": [\n 1523\n ]\n },\n \"serviceName\": {\n \"type\": \"string\",\n \"description\": \"The name of the service.\",\n \"examples\": [\n \"service\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"port\",\n \"serviceName\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"KafkaServer\",\n \"description\": \"Kafka Server\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"kafka\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The bootstrap server of the kafka cluster.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the message. Examples: json, avro, protobuf. Default: json.\",\n \"default\": \"json\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"topic\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"PubSubServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"pubsub\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"project\",\n \"topic\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"KinesisDataStreamsServer\",\n \"description\": \"Kinesis Data Streams Server\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"kinesis\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"stream\": {\n \"type\": \"string\",\n \"description\": \"The name of the Kinesis data stream.\"\n },\n \"region\": {\n \"type\": \"string\",\n \"description\": \"AWS region.\",\n \"examples\": [\n \"eu-west-1\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the record\",\n \"examples\": [\n \"json\",\n \"avro\",\n \"protobuf\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"stream\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"TrinoServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"const\": \"trino\",\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the catalog.\",\n \"examples\": [\n \"hive\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"my_schema\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"host\",\n \"port\",\n \"catalog\",\n \"schema\"\n ]\n },\n {\n \"type\": \"object\",\n \"title\": \"LocalServer\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"local\"\n ],\n \"description\": \"The type of the data product technology that implements the data contract.\"\n },\n \"path\": {\n \"type\": \"string\",\n \"description\": \"The relative or absolute path to the data file(s).\",\n \"examples\": [\n \"./folder/data.parquet\",\n \"./folder/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the file(s)\",\n \"examples\": [\n \"json\",\n \"parquet\",\n \"delta\",\n \"csv\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\",\n \"path\",\n \"format\"\n ]\n }\n ]\n },\n \"description\": \"Information about the servers.\"\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n },\n \"additionalProperties\": true\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Model\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, view, object. Default: table.\",\n \"type\": \"string\",\n \"title\": \"ModelType\",\n \"default\": \"table\",\n \"enum\": [\n \"table\",\n \"view\",\n \"object\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the model. Especially useful if the model name is cryptic or contains abbreviations.\",\n \"examples\": [\"Purchase Orders\", \"Air Shipments\"]\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Field\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"required\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if this field must contain a value and may not be null.\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"primary\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"If this field is a primary key.\"\n },\n \"references\": {\n \"type\": \"string\",\n \"description\": \"The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship.\",\n \"examples\": [\n \"orders.order_id\",\n \"model.nested_field.field\"\n ]\n },\n \"unique\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if the value must be unique within the model.\"\n },\n \"enum\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"uniqueItems\": true,\n \"description\": \"A value must be equal to one of the elements in this array value. Only evaluated if the value is not null.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must greater than, or equal to, the value of this. Only applies to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must less than, or equal to, the value of this. Only applies to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"A specific format the value must comply with (e.g., 'email', 'uri', 'uuid').\",\n \"examples\": [\n \"email\",\n \"uri\",\n \"uuid\"\n ]\n },\n \"precision\": {\n \"type\": \"number\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"number\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression the value must match. Only applies to string types.\",\n \"examples\": [\n \"^[a-zA-Z0-9_-]+$\"\n ]\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\n \"sensitive\",\n \"restricted\",\n \"internal\",\n \"public\"\n ]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for field configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroType\": {\n \"type\": \"string\",\n \"description\": \"Specify the field type to use when exporting the data model to Apache Avro.\"\n },\n \"avroLogicalType\": {\n \"type\": \"string\",\n \"description\": \"Specify the logical field type to use when exporting the data model to Apache Avro.\"\n },\n \"bigqueryType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)`.\"\n },\n \"snowflakeType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Snowflake table, e.g., `TIMESTAMP_LTZ`.\"\n },\n \"redshiftType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Redshift table, e.g., `SMALLINT`.\"\n },\n \"sqlserverType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a SQL Server table, e.g., `DATETIME2`.\"\n },\n \"databricksType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Databricks Unity Catalog table.\"\n },\n \"glueType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in an AWS Glue Data Catalog table.\"\n }\n }\n }\n }\n }\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for model configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n\n\n ]\n },\n \"properties\": {\n \"avroNamespace\": {\n \"type\": \"string\",\n \"description\": \"The namespace to use when importing and exporting the data model from / to Apache Avro.\"\n }\n }\n }\n }\n }\n },\n \"definitions\": {\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"type\": \"object\",\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Definition\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"name\",\n \"type\"\n ]\n }\n },\n \"schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"title\": \"SchemaType\",\n \"enum\": [\n \"dbt\",\n \"bigquery\",\n \"json-schema\",\n \"sql-ddl\",\n \"avro\",\n \"protobuf\",\n \"custom\"\n ],\n \"description\": \"The type of the schema. Typical values are dbt, bigquery, json-schema, sql-ddl, avro, protobuf, custom.\"\n },\n \"specification\": {\n \"oneOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the schema as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the schema as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The schema of the data contract describes the syntax and semantics of provided data sets. It supports different schema types.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"title\": \"ExampleType\",\n \"enum\": [\n \"csv\",\n \"json\",\n \"yaml\",\n \"custom\"\n ],\n \"description\": \"The type of the example data. Well-known types are csv, json, yaml, custom.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the example.\"\n },\n \"model\": {\n \"type\": \"string\",\n \"description\": \"The reference to the model in the schema, e.g., a table name.\"\n },\n \"data\": {\n \"oneOf\": [\n {\n \"type\": \"string\",\n \"description\": \"Example data for this model.\"\n },\n {\n \"type\": \"array\",\n \"description\": \"Example data for this model in a structured format. Use this for type json or yaml.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"data\"\n ]\n },\n \"description\": \"The Examples Object is an array of Example Objects.\"\n },\n \"servicelevels\": {\n \"type\": \"object\",\n \"description\": \"Specifies the service level agreements for the provided data, including availability, data retention policies, latency requirements, data freshness, update frequency, support availability, and backup policies.\",\n \"properties\": {\n \"availability\": {\n \"type\": \"object\",\n \"description\": \"Availability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the availability service level.\",\n \"example\": \"The server is available during support hours\"\n },\n \"percentage\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the guaranteed uptime in percent (e.g., `99.9%`)\",\n \"pattern\": \"^\\\\d+(\\\\.\\\\d+)?%$\",\n \"example\": \"99.9%\"\n }\n }\n },\n \"retention\": {\n \"type\": \"object\",\n \"description\": \"Retention covers the period how long data will be available.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the retention service level.\",\n \"example\": \"Data is retained for one year.\"\n },\n \"period\": {\n \"type\": \"string\",\n \"description\": \"An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`).\",\n \"example\": \"P1Y\"\n },\n \"unlimited\": {\n \"type\": \"boolean\",\n \"description\": \"An optional indicator that data is kept forever.\",\n \"example\": false\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the period refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"latency\": {\n \"type\": \"object\",\n \"description\": \"Latency refers to the maximum amount of time from the source to its destination.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the latency service level.\",\n \"example\": \"Data is available within 25 hours after the order was placed.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`).\",\n \"example\": \"25h\"\n },\n \"sourceTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp when the data was provided at the source.\",\n \"example\": \"orders.order_timestamp\"\n },\n \"processedTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract.\",\n \"example\": \"orders.processed_timestamp\"\n }\n }\n },\n \"freshness\": {\n \"type\": \"object\",\n \"description\": \"The maximum age of the youngest row in a table.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the freshness service level.\",\n \"example\": \"The age of the youngest row in a table is within 25 hours.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g., `PT24H`).\",\n \"example\": \"25h\"\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the threshold refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"frequency\": {\n \"type\": \"object\",\n \"description\": \"Frequency describes how often data is updated.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the frequency service level.\",\n \"example\": \"Data is delivered once a day.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"batch\",\n \"micro-batching\",\n \"streaming\",\n \"manual\"\n ],\n \"description\": \"The method of data processing.\",\n \"example\": \"batch\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`.\",\n \"example\": \"daily\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * *\"\n }\n }\n },\n \"support\": {\n \"type\": \"object\",\n \"description\": \"Support describes the times when support will be available for contact.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the support service level.\",\n \"example\": \"The data is available during typical business hours at headquarters.\"\n },\n \"time\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`.\",\n \"example\": \"9am to 5pm in EST on business days\"\n },\n \"responseTime\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with.\",\n \"example\": \"24 hours\"\n }\n }\n },\n \"backup\": {\n \"type\": \"object\",\n \"description\": \"Backup specifies details about data backup procedures.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the backup service level.\",\n \"example\": \"Data is backed up once a week, every Sunday at 0:00 UTC.\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"An optional interval that defines how often data will be backed up, e.g., `daily`.\",\n \"example\": \"weekly\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"An optional cron expression when data will be backed up, e.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * 0\"\n },\n \"recoveryTime\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours).\",\n \"example\": \"24 hours\"\n },\n \"recoveryPoint\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours).\",\n \"example\": \"1 week\"\n }\n }\n }\n }\n },\n \"quality\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"title\": \"QualityType\",\n \"enum\": [\n \"SodaCL\",\n \"montecarlo\",\n \"great-expectations\",\n \"custom\"\n ],\n \"description\": \"The type of the quality check. Typical values are SodaCL, montecarlo, great-expectations, custom.\"\n },\n \"specification\": {\n \"oneOf\": [\n {\n \"type\": \"string\",\n \"description\": \"The specification of the quality attributes as a string.\"\n },\n {\n \"type\": \"object\",\n \"description\": \"The specification of the quality attributes as an object.\"\n }\n ]\n }\n },\n \"required\": [\n \"type\",\n \"specification\"\n ],\n \"description\": \"The quality object contains quality attributes and checks.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"description\": \"Tags to facilitate searching and filtering.\",\n \"examples\": [\n \"databricks\",\n \"pii\",\n \"sensitive\"\n ]\n },\n \"description\": \"Tags to facilitate searching and filtering.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ],\n \"$defs\": {\n \"FieldType\": {\n \"type\": \"string\",\n \"title\": \"FieldType\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\",\n \"decimal\",\n \"numeric\",\n \"int\",\n \"integer\",\n \"long\",\n \"bigint\",\n \"float\",\n \"double\",\n \"string\",\n \"text\",\n \"varchar\",\n \"boolean\",\n \"timestamp\",\n \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"array\",\n \"map\",\n \"object\",\n \"record\",\n \"struct\",\n \"bytes\",\n \"null\"\n ]\n }\n }\n}" }, { "path": "versions/0.9.3/definition.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"name\",\n \"type\"\n ]\n}" }, { "path": "versions/1.1.0/README.md", "content": "# Data Contract Specification\n\n\n \"Stars\"\n\"Slack\n\n![datacontract.png](images/datacontract.png)\n\nData contracts bring data providers and data consumers together.\n\nA _data contract_ is a document that defines the structure, format, semantics, quality, and terms of use for exchanging data between a data provider and their consumers.\nThink of an API, but for data.\nA data contract is implemented by a data product or other data technologies, even legacy data warehouses.\nData contracts can also be used for the input port to specify the expectations of data dependencies and verify given guarantees.\n\nThe _data contract specification_ defines a YAML format to describe attributes of provided data sets.\nIt is data platform neutral and can be used with any data platform, such as AWS S3, Google BigQuery, Azure, Databricks, and Snowflake.\nThe data contract specification is an open initiative to define a common data contract format.\nIt follows [OpenAPI](https://www.openapis.org/) and [AsyncAPI](https://www.asyncapi.com/) conventions.\n\nData contracts come into play when data is exchanged between different teams or organizational units, such as in a [data mesh architecture](https://www.datamesh-architecture.com/).\nFirst, and foremost, data contracts are a communication tool to express a common understanding of how data should be structured and interpreted.\nThey make semantic and quality expectations explicit.\nThey are often created collaboratively in [workshops](./workshop.md) together with data providers and data consumers.\nLater in development and production, they also serve as the basis for code generation, testing, schema validations, quality checks, monitoring, access control, and computational governance policies.\n\nThe specification comes along with the [Data Contract CLI](https://github.com/datacontract/datacontract-cli), an open-source tool to develop, validate, and enforce data contracts.\n\n> _Note: The term \"data contract\" refers to a specification that is usually owned by the data provider and thus does not align with a \"contract\" in a legal sense as a mutual agreement between two parties.\n> The term \"contract\" may be somewhat misleading, but it is how it is used by the industry.\n> The mutual agreement between one data provider and one data consumer is the \"data usage agreement\" that refers to a data contract.\n> Data usage agreements have a defined lifecycle, start/end date, and help the data provider to track who accesses their data and for which purposes._\n\nVersion\n---\n\n1.1.0([Changelog](CHANGELOG.md))\n\nExample\n---\n\nView in [Data Contract Catalog](https://datacontract.com/examples/index.html)\n\n```yaml\ndataContractSpecification: 1.1.0\nid: urn:datacontract:checkout:orders-latest\ninfo:\n title: Orders Latest\n version: 2.0.0\n description: |\n Successful customer orders in the webshop.\n All orders since 2020-01-01.\n Orders with their line items are in their current state (no history included).\n owner: Checkout Team\n status: active\n contact:\n name: John Doe (Data Product Owner)\n url: https://teams.microsoft.com/l/channel/example/checkout\nservers:\n production:\n type: s3\n environment: prod\n location: s3://datacontract-example-orders-latest/v2/{model}/*.json\n format: json\n delimiter: new_line\n description: \"One folder per model. One file per day.\"\n roles:\n - name: analyst_us\n description: Access to the data for US region\n - name: analyst_cn\n description: Access to the data for China region\nterms:\n usage: |\n Data can be used for reports, analytics and machine learning use cases.\n Order may be linked and joined by other tables\n limitations: |\n Not suitable for real-time use cases.\n Data may not be used to identify individual customers.\n Max data processing per day: 10 TiB\n policies:\n - name: privacy-policy\n url: https://example.com/privacy-policy\n - name: license\n description: External data is licensed under agreement 1234.\n url: https://example.com/license/1234\n billing: 5000 USD per month\n noticePeriod: P3M\nmodels:\n orders:\n description: One record per order. Includes cancelled and deleted orders.\n type: table\n fields:\n order_id:\n $ref: '#/definitions/order_id'\n required: true\n unique: true\n primaryKey: true\n order_timestamp:\n description: The business timestamp in UTC when the order was successfully registered in the source system and the payment was successful.\n type: timestamp\n required: true\n examples:\n - \"2024-09-09T08:30:00Z\"\n tags: [\"business-timestamp\"]\n order_total:\n description: Total amount the smallest monetary unit (e.g., cents).\n type: long\n required: true\n examples:\n - 9999\n quality:\n - type: sql\n description: 95% of all order total values are expected to be between 10 and 499 EUR.\n query: |\n SELECT quantile_cont(order_total, 0.95) AS percentile_95\n FROM orders\n mustBeBetween: [1000, 49900]\n customer_id:\n description: Unique identifier for the customer.\n type: text\n minLength: 10\n maxLength: 20\n customer_email_address:\n description: The email address, as entered by the customer.\n type: text\n format: email\n required: true\n pii: true\n classification: sensitive\n quality:\n - type: text\n description: The email address is not verified and may be invalid.\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: email_address\n processed_timestamp:\n description: The timestamp when the record was processed by the data platform.\n type: timestamp\n required: true\n config:\n jsonType: string\n jsonFormat: date-time\n quality:\n - type: sql\n description: The maximum duration between two orders should be less that 3600 seconds\n query: |\n SELECT MAX(duration) AS max_duration \n FROM (\n SELECT EXTRACT(EPOCH FROM (order_timestamp - LAG(order_timestamp) OVER (ORDER BY order_timestamp))) AS duration \n FROM orders\n )\n mustBeLessThan: 3600\n - type: sql\n description: Row Count\n query: |\n SELECT count(*) as row_count\n FROM orders\n mustBeGreaterThan: 5\n examples:\n - |\n order_id,order_timestamp,order_total,customer_id,customer_email_address,processed_timestamp\n \"1001\",\"2030-09-09T08:30:00Z\",2500,\"1000000001\",\"mary.taylor82@example.com\",\"2030-09-09T08:31:00Z\"\n \"1002\",\"2030-09-08T15:45:00Z\",1800,\"1000000002\",\"michael.miller83@example.com\",\"2030-09-09T08:31:00Z\"\n \"1003\",\"2030-09-07T12:15:00Z\",3200,\"1000000003\",\"michael.smith5@example.com\",\"2030-09-09T08:31:00Z\"\n \"1004\",\"2030-09-06T19:20:00Z\",1500,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1005\",\"2030-09-05T10:10:00Z\",4200,\"1000000004\",\"elizabeth.moore80@example.com\",\"2030-09-09T08:31:00Z\"\n \"1006\",\"2030-09-04T14:55:00Z\",2800,\"1000000005\",\"john.davis28@example.com\",\"2030-09-09T08:31:00Z\"\n \"1007\",\"2030-09-03T21:05:00Z\",1900,\"1000000006\",\"linda.brown67@example.com\",\"2030-09-09T08:31:00Z\"\n \"1008\",\"2030-09-02T17:40:00Z\",3600,\"1000000007\",\"patricia.smith40@example.com\",\"2030-09-09T08:31:00Z\"\n \"1009\",\"2030-09-01T09:25:00Z\",3100,\"1000000008\",\"linda.wilson43@example.com\",\"2030-09-09T08:31:00Z\"\n \"1010\",\"2030-08-31T22:50:00Z\",2700,\"1000000009\",\"mary.smith98@example.com\",\"2030-09-09T08:31:00Z\"\n line_items:\n description: A single article that is part of an order.\n type: table\n fields:\n line_item_id:\n type: text\n description: Primary key of the lines_item_id table\n required: true\n order_id:\n $ref: '#/definitions/order_id'\n references: orders.order_id\n sku:\n description: The purchased article number\n $ref: '#/definitions/sku'\n primaryKey: [\"order_id\", \"line_item_id\"]\n examples:\n - |\n line_item_id,order_id,sku\n \"LI-1\",\"1001\",\"5901234123457\"\n \"LI-2\",\"1001\",\"4001234567890\"\n \"LI-3\",\"1002\",\"5901234123457\"\n \"LI-4\",\"1002\",\"2001234567893\"\n \"LI-5\",\"1003\",\"4001234567890\"\n \"LI-6\",\"1003\",\"5001234567892\"\n \"LI-7\",\"1004\",\"5901234123457\"\n \"LI-8\",\"1005\",\"2001234567893\"\n \"LI-9\",\"1005\",\"5001234567892\"\n \"LI-10\",\"1005\",\"6001234567891\"\ndefinitions:\n order_id:\n title: Order ID\n type: text\n format: uuid\n description: An internal ID that identifies an order in the online shop.\n examples:\n - 243c25e5-a081-43a9-aeab-6d5d5b6cb5e2\n pii: true\n classification: restricted\n tags:\n - orders\n sku:\n title: Stock Keeping Unit\n type: text\n pattern: ^[A-Za-z0-9]{8,14}$\n examples:\n - \"96385074\"\n description: |\n A Stock Keeping Unit (SKU) is an internal unique identifier for an article.\n It is typically associated with an article's barcode, such as the EAN/GTIN.\n links:\n wikipedia: https://en.wikipedia.org/wiki/Stock_keeping_unit\n tags:\n - inventory\nservicelevels:\n availability:\n description: The server is available during support hours\n percentage: 99.9%\n retention:\n description: Data is retained for one year\n period: P1Y\n unlimited: false\n latency:\n description: Data is available within 25 hours after the order was placed\n threshold: 25h\n sourceTimestampField: orders.order_timestamp\n processedTimestampField: orders.processed_timestamp\n freshness:\n description: The age of the youngest row in a table.\n threshold: 25h\n timestampField: orders.order_timestamp\n frequency:\n description: Data is delivered once a day\n type: batch # or streaming\n interval: daily # for batch, either or cron\n cron: 0 0 * * * # for batch, either or interval\n support:\n description: The data is available during typical business hours at headquarters\n time: 9am to 5pm in EST on business days\n responseTime: 1h\n backup:\n description: Data is backed up once a week, every Sunday at 0:00 UTC.\n interval: weekly\n cron: 0 0 * * 0\n recoveryTime: 24 hours\n recoveryPoint: 1 week\ntags:\n - checkout\n - orders\n - s3\nlinks:\n datacontractCli: https://cli.datacontract.com\n```\n\nData Contract CLI\n---\n\nThe [Data Contract CLI](https://cli.datacontract.com) is a command line tool and Python library to lint, test, import and export data contracts.\n\nHere is short example how to verify that your actual dataset matches the data contract:\n\n```bash\npip3 install \"datacontract-cli[all]\"\ndatacontract test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nor, if you prefer Docker:\n```bash\ndocker run datacontract/cli test https://datacontract.com/examples/orders-latest/datacontract.yaml\n```\n\nThe Data Contract contains all required information to verify data:\n\n- The _servers_ block has the connection details to the actual data set.\n- The _models_ define the syntax, formats, and constraints.\n- The _quality_ defined further quality checks.\n\nThe Data Contract CLI chooses the appropriate engine, formulates test cases, connects to the server, and executes the tests, based on the server type.\n\nMore information and configuration options on [cli.datacontract.com](https://cli.datacontract.com).\n\n\nSpecification\n---\n\n![The eight major categories in the data contract specification](images/categories.png)\n\n- [Data Contract Object](#data-contract-object)\n- [Info Object](#info-object)\n- [Contact Object](#contact-object)\n- [Server Object](#server-object)\n- [Terms Object](#terms-object)\n- [Model Object](#model-object)\n- [Field Object](#field-object)\n- [Definition Object](#definition-object)\n- [Service Level Object](#service-levels-object)\n- [Quality Object](#quality-object)\n- [Lineage Object](#lineage-object)\n- [Data Types](#data-types)\n- [Specification Extensions](#specification-extensions)\n\n\n[JSON Schema](https://github.com/datacontract/datacontract-specification/blob/main/datacontract.schema.json) of the Data Contract Specification.\n\n### Data Contract Object\n\nThis is the root document.\n\nIt is _RECOMMENDED_ that the root document be named: `datacontract.yaml`.\n\n| Field | Type | Description |\n|---------------------------|--------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|\n| dataContractSpecification | `string` | REQUIRED. Specifies the Data Contract Specification being used. |\n| id | `string` | REQUIRED. An organization-wide unique technical identifier, such as a UUID, URN, slug, string, or number |\n| info | [Info Object](#info-object) | REQUIRED. Specifies the metadata of the data contract. May be used by tooling. |\n| servers | Map[`string`, [Server Object](#server-object)] | Specifies the servers of the data contract. |\n| terms | [Terms Object](#terms-object) | Specifies the terms and conditions of the data contract. |\n| models | Map[`string`, [Model Object](#model-object)] | Specifies the logical data model. |\n| definitions | Map[`string`, [Definition Object](#definition-object)] | Specifies definitions. |\n| servicelevels | [Service Levels Object](#service-levels-object) | Specifies the service level of the provided data |\n| links | Map[`string`, `string`] | Additional external documentation links. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n\n### Info Object\n\nMetadata and life cycle information about the data contract.\n\n\n| Field | Type | Description |\n|-------------|-----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| title | `string` | REQUIRED. The title of the data contract. |\n| version | `string` | REQUIRED. The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version). |\n| status | `string` | The status of the data contract. Can be `proposed`, `in development`, `active`, `deprecated`, `retired`. |\n| description | `string` | A description of the data contract. |\n| owner | `string` | The owner or team responsible for managing the data contract and providing the data. |\n| contact | [Contact Object](#contact-object) | Contact information for the data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Contact Object\n\nContact information for the data contract.\n\n| Field | Type | Description |\n|-------|----------|-------------------------------------------------------------------------------------------------------|\n| name | `string` | The identifying name of the contact person/organization. |\n| url | `string` | The URL pointing to the contact information. This _MUST_ be in the form of a URL. |\n| email | `string` | The email address of the contact person/organization. This _MUST_ be in the form of an email address. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n### Server Object\n\nThe fields are dependent on the defined type.\n\n| Field | Type | Description |\n|-------------|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | REQUIRED. The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `s3`, `glue`, `redshift`, `azure`, `sqlserver`, `snowflake`, `databricks`, `postgres`, `oracle`, `kafka`, `pubsub`, `sftp`, `kinesis`, `trino`, `local` |\n| description | `string` | An optional string describing the server. |\n| environment | `string` | An optional string describing the environment, e.g., prod, sit, stg. |\n| roles | Array of [Server Role Object](#server-role-object) | An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### BigQuery Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `bigquery` |\n| project | `string` | The GCP project name. |\n| dataset | `string` | |\n\n#### S3 Server Object\n\n| Field | Type | Description |\n|-------------|----------|-------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `s3` |\n| location | `string` | S3 URL, starting with `s3://` |\n| endpointUrl | `string` | The server endpoint for S3-compatible servers, such as MioIO or Google Cloud Storage, e.g., `https://minio.example.com` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\nExample (AWS S3):\n\n```yaml\nservers:\n production:\n type: s3\n location: s3://acme-orders-prod/orders/\n format: json\n delimiter: new_line\n```\n\nExample (MinIO):\n\n```yaml\nservers:\n minio:\n type: s3\n endpointUrl: http://localhost:9000\n location: s3://my-bucket/path/\n format: delta\n```\n\nExample (Google Cloud Storage):\n\n```yaml\nservers:\n gcs:\n type: s3\n endpointUrl: https://storage.googleapis.com\n location: s3://my-bucket/path/*/*/*/*/*.parquet\n format: parquet\n```\n\n\n#### Redshift Server Object\n\n| Field | Type | Description |\n|-------------------|----------|---------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `redshift` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n| clusterIdentifier | `string` | Identifier of the cluster.
Example: `analytics-cluster` |\n| host | `string` | Host of the cluster.
Example: `analytics-cluster.example.eu-west-1.redshift.amazonaws.com` |\n| port | `number` | Port of the cluster.
Example: `5439` |\n| endpoint | `string` | Endpoint of the cluster
Example: `analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics` |\n\nExample, specifying an endpoint:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n endpoint: analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\n```\n\nExample, specifying the cluster identifier:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n clusterIdentifier: analytics-cluster\n```\n\nExample, specifying the cluster host:\n\n```yaml\nservers:\n analytics:\n type: redshift\n account: '123456789012'\n database: analytics\n schema: analytics\n host: analytics-cluster.example.eu-west-1.redshift.amazonaws.com\n port: 5439\n```\n\n#### Azure Server Object\n\n| Field | Type | Description |\n|----------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `azure` |\n| storageAccount | `string` | The storage account name that contains the files |\n| location | `string` | Path to Azure Blob Storage or Azure Data Lake Storage (ADLS) in the storage account, supports globs. Starting with `az://` or `abfss`
Recommended pattern is `abfss:///`, Examples: `az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet` or `abfss://my_container_name/path/*.parquet` |\n| format | `string` | Format of files, such as `parquet`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\n\n#### SQL-Server Server Object\n\n| Field | Type | Description |\n|----------|-----------|--------------------------------------------------------------------------|\n| type | `string` | `sqlserver` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server, default: `1433` |\n| database | `string` | The name of the database, e.g., `database`. |\n| schema | `string` | The name of the schema in the database, e.g., `dbo`. |\n| driver | `string` | The name of the supported driver, e.g., `ODBC Driver 18 for SQL Server`. |\n\n\n#### Snowflake Server Object\n\n| Field | Type | Description |\n|----------|----------|-------------|\n| type | `string` | `snowflake` |\n| account | `string` | |\n| database | `string` | |\n| schema | `string` | |\n\n#### Databricks Server Object\n\n| Field | Type | Description |\n|---------|----------|---------------------------------------------------------------------|\n| type | `string` | `databricks` |\n| host | `string` | The Databricks host, e.g., `dbc-abcdefgh-1234.cloud.databricks.com` |\n| catalog | `string` | The name of the Hive or Unity catalog |\n| schema | `string` | The schema name in the catalog |\n\n#### Postgres Server Object\n\n| Field | Type | Description |\n|----------|-----------|---------------------------------------------------------|\n| type | `string` | `postgres` |\n| host | `string` | The host to the database server |\n| port | `integer` | The port to the database server |\n| database | `string` | The name of the database, e.g., `postgres`. |\n| schema | `string` | The name of the schema in the database, e.g., `public`. |\n\n#### Oracle Server Object\n\n| Field | Type | Description |\n|-------------|-----------|---------------------------------|\n| type | `string` | `oracle` |\n| host | `string` | The host to the oracle server |\n| port | `integer` | The port to the oracle server |\n| serviceName | `string` | The name of the service |\n\n#### Kafka Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kafka` |\n| host | `string` | The bootstrap server of the kafka cluster. |\n| topic | `string` | The topic name. |\n| format | `string` | The format of the message. Examples: json, avro, protobuf. Default: json. |\n\n#### Pub/Sub Server Object\n\n| Field | Type | Description |\n|---------|----------|-----------------------|\n| type | `string` | `pubsub` |\n| project | `string` | The GCP project name. |\n| topic | `string` | The topic name. |\n\n#### sftp Server Object\n\n| Field | Type | Description |\n|-----------|----------|------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `sftp` |\n| location | `string` | S3 URL, starting with `sftp://` |\n| format | `string` | Format of files, such as `parquet`, `delta`, `json`, `csv` |\n| delimiter | `string` | (Only for format = `json`), how multiple json documents are delimited within one file, e.g., `new_line`, `array` |\n\n#### AWS Kinesis Data Streams Server Object\n\n| Field | Type | Description |\n|--------|----------|---------------------------------------------------------------------------|\n| type | `string` | `kinesis` |\n| stream | `string` | The name of the Kinesis data stream. |\n| region | `string` | AWS region, e.g., `eu-west-1`. |\n| format | `string` | The format of the records. Examples: json, avro, protobuf. |\n\n#### Trino Server Object\n\n| Field | Type | Description |\n|----------|-----------|-----------------------------------------------------------|\n| type | `string` | `trino` |\n| host | `string` | The Trino host |\n| port | `integer` | The Trino port | \n| catalog | `string` | The name of the catalog, e.g., `my_catalog`. |\n| schema | `string` | The name of the schema in the catalog, e.g., `my_schema`. |\n\n#### Local Server Object\n\n| Field | Type | Description |\n|--------|----------|-------------------------------------------------------------------------------------|\n| type | `string` | `local` |\n| path | `string` | The relative or absolute path to the data file(s), such as `./folder/data.parquet`. |\n| format | `string` | The format of the file(s), such as `parquet`, `delta`, `csv`, or `json`. |\n\n#### Server Role Object\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------|\n| name | `string` | Name of the role |\n| description | `string` | A description of the role and what access the role provides. |\n\n### Terms Object\n\nThe terms and conditions of the data contract.\n\n| Field | Type | Description |\n|--------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| usage | `string` | The usage describes the way the data is expected to be used. Can contain business and technical information. |\n| limitations | `string` | The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for. |\n| policies | Array of [Policy Object](#policy-object) | A list of policies, licenses, standards, that are applicable for this data contract and that must be acknowledged by data consumers. |\n| billing | `string` | The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use. |\n| noticePeriod | `string` | The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., `P3M` for a period of three months. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Policy Object\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------|\n| name | `string` | Name of the policy. |\n| description | `string` | A description of the policy. |\n| url | `string` | An URL that refers to the policy. |\n\n\n### Model Object\n\nThe Model Object describes the structure and semantics of a data model, such as tables, views, or structured files.\n\nThe name of the data model (table name) is defined by the key that refers to this Model Object.\n\n| Field | Type | Description |\n|-------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | The type of the model. Examples: `table`, `view`, `object`. Default: `table`. |\n| description | `string` | An optional string describing the data model. |\n| title | `string` | An optional string for the title of the data model. Especially useful if the name of the model is cryptic or contains abbreviations. |\n| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |\n| primaryKey | Array of `string` | If the primary key is a compound key, list the field names that constitute the primary key. Alternative to field-level `primaryKey`. |\n| quality | Array of [Quality Object](#quality-object) | Specifies the quality attributes on model level. |\n| examples | Array of `Any` | Specifies example data sets for the model. |\n| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |\n\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Field Object\n\nThe Field Objects describes one field (column, property, nested field) of a data model.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the semantic of the data in this field. |\n| type | [Data Type](#data-types) | The logical data type of the field. |\n| title | `string` | An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations. |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| required | `boolean` | An indication, if this field must contain a value and may not be null. Default: `false` |\n| primaryKey | `boolean` | If this field is a primary key. Default: `false` |\n| references | `string` | The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship. |\n| unique | `boolean` | An indication, if the value must be unique within the model. Default: `false` |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| precision | `number` | The maximum number of digits in a number. Only applies to numeric values. Defaults to 38. |\n| scale | `number` | The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0. |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| ~~example~~ | `string` | DEPRECATED, use examples. An example value. |\n| examples | Array of Any | A list of example values. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). | \n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. Examples may be: `sensitive`, `restricted`, `internal`, `public`. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| links | Map[`string`,`string`] | Additional external documentation links. |\n| $ref | `string` | A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition. |\n| fields | Map[`string`, [Field Object](#field-object)] | The nested fields (e.g. columns) of the object, record, or struct. Use only when type is `object`, `record`, or `struct`. |\n| items | [Field Object](#field-object) | The type of the elements in the array. Use only when type is `array`. |\n| keys | [Field Object](#field-object) | Describes the key structure of a map. Defaults to `type: string` if a map is defined as type. Not all server types support different key types. Use only when type is `map`. |\n| values | [Field Object](#field-object) | Describes the value structure of a map. Use only when type is `map`. |\n| quality | Array of [Quality Object](#quality-object) | Specifies the quality attributes on field level. |\n| lineage | [Lineage Object](#lineage-object) | Provides information where the data comes from. |\n| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n### Definition Object\n\nThe Definition Object includes a clear and concise explanations of syntax, semantic, and classification of a business object in a given domain.\nIt serves as a reference for a common understanding of terminology, ensure consistent usage and to identify join-able fields.\nModels fields can refer to definitions using the `$ref` field to link to existing definitions and avoid duplicate documentations.\n\n| Field | Type | Description |\n|------------------|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | [Data Type](#data-types) | REQUIRED. The logical data type |\n| title | `string` | The business name of this definition. |\n| description | `string` | Clear and concise explanations related to the domain |\n| enum | array of `string` | A value must be equal to one of the elements in this array value. Only evaluated if the value is not null. |\n| format | `string` | `email`: A value must be complaint to [RFC 5321, section 4.1.2](https://www.rfc-editor.org/info/rfc5321).
`uri`: A value must be complaint to [RFC 3986](https://www.rfc-editor.org/info/rfc3986).
`uuid`: A value must be complaint to [RFC 4122](https://www.rfc-editor.org/info/rfc4122). Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| precision | `number` | The maximum number of digits in a number. Only applies to numeric values. Defaults to 38. |\n| scale | `number` | The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0. |\n| minLength | `number` | A value must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| maxLength | `number` | A value must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| pattern | `string` | A value must be valid according to the [ECMA-262](https://262.ecma-international.org/5.1/) regular expression dialect. Only evaluated if the value is not null. Only applies to unicode character sequences types (`string`, `text`, `varchar`). |\n| minimum | `number` | A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMinimum | `number` | A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| maximum | `number` | A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| exclusiveMaximum | `number` | A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values. |\n| examples | Array of Any | A list of example values. |\n| pii | `boolean` | An indication, if this field contains Personal Identifiable Information (PII). |\n| classification | `string` | The data class defining the sensitivity level for this field, according to the organization's classification scheme. |\n| tags | Array of `string` | Custom metadata to provide additional context. |\n| links | Map[`string`, `string`] | Additional external documentation links. |\n| fields | Map[`string`, [Field Object](#field-object)] | The nested fields (e.g. columns) of the object, record, or struct. Use only when type is `object`, `record`, or `struct`. |\n| items | [Field Object](#field-object) | The type of the elements in the array. Use only when type is `array`. |\n| keys | [Field Object](#field-object) | Describes the key structure of a map. Defaults to `type: string` if a map is defined as type. Not all server types support different key types. Use only when type is `map`. |\n| values | [Field Object](#field-object) | Describes the value structure of a map. Use only when type is `map`. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n\n### Service Levels Object\n\nA service level is defined as an agreed-upon, measurable level of performance for provided the data.\nData Contract Specification defines well-known service levels.\nThis list can be extended with custom service levels.\n\nOne can either describe each service level informally using the `description` field, or make use of the predefined fields for automation support, e.g., via the [Data Contract CLI](https://cli.datacontract.com).\n\n| Field | Type | Description |\n|--------------|-----------------------------------------------|-------------------------------------------------------------------------|\n| availability | [Availability Object](#availability-object) | The promised uptime of the system that provides the data |\n| retention | [Retention Object](#retention-object) | The period how long data will be available. |\n| latency | [Latency Object](#latency-object) | The maximum amount of time from the source to its destination. |\n| freshness | [Freshness Object](#freshness-object) | The maximum age of the youngest entry. |\n| frequency | [Frequency Object](#frequency-object) | The update frequency. |\n| support | [Support Object](#support-object) | The times when support is provided. |\n| backup | [Backup Object](#backup-object) | The details about data backup procedures. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Availability Object\n\nAvailability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the availability service level. |\n| percentage | `string` | An optional string describing the guaranteed uptime in percent (e.g., `99.9%`) |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Retention Object\n\nRetention covers the period how long data will be available.\n\n| Field | Type | Description |\n|----------------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the retention service level. |\n| period | `string` | An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`). |\n| unlimited | `boolean` | An optional indicator that data is kept forever. |\n| timestampField | `string` | An optional reference to the field that contains the timestamp that the period refers to. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Latency Object\n\nLatency refers to the maximum amount of time from the source to its destination.\n\nExamples are the maximum duration it takes after an order has been recorded in the ecommerce shop until it is available in the orders table in the data analytics platform. This includes the waiting times until the next batch run is started and the processing time of the pipeline.\n\n| Field | Type | Description |\n|-------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the latency service level. |\n| threshold | `string` | An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`). |\n| sourceTimestampField | `string` | An optional reference to the field that contains the timestamp when the data was provided at the source. |\n| processedTimestampField | `string` | An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Freshness Object\n\nFreshness refers to the maximum age of the youngest entry.\n\n| Field | Type | Description |\n|-------------------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the freshness service level. |\n| threshold | `string` | An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`). |\n| timestampField | `string` | An optional reference to the field that contains the timestamp that the threshold refers to. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n#### Frequency Object\n\nFrequency describes how often data is updated.\n\n| Field | Type | Description |\n|-------------|----------|-----------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the frequency service level. |\n| type | `string` | An optional type of data processing. Typical values are `batch`, `micro-batching`, `streaming`, `manual`. |\n| interval | `string` | Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`. |\n| cron | `string` | Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`. |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n#### Support Object\n\nSupport describes the times when support will be available for contact.\n\n| Field | Type | Description |\n|--------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the support service level. |\n| time | `string` | An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`. |\n| responseTime | `string` | An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with. |\n\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\n\n#### Backup Object\n\nBackup specifies details about data backup procedures.\n\n| Field | Type | Description |\n|---------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| description | `string` | An optional string describing the backup service level. |\n| interval | `string` | An optional interval that defines how often data will be backed up, e.g., `daily`. |\n| cron | `string` | An optional cron expression when data will be backed up, e.g., `0 0 * * *`. |\n| recoveryTime | `string` | An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours). |\n| recoveryPoint | `string` | An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours). |\n\n\n\n### Quality Object\n\nThe quality object defines quality attributes.\n\nQuality attributes are checks that can be applied to the data to ensure its quality.\nData can be verified by executing these checks through a data quality engine.\n\nQuality attributes can be:\n- A text in natural language that describes the quality of the data.\n- An individual SQL query that returns a single value that can be compared.\n- Engine-specific types: Pre-defined quality checks, as defined by data quality libraries. Currently, the engines `soda` and `great-expectations` are supported.\n\nA quality object can be specified on field level and on model level.\nThe top-level quality object is deprecated.\n\n#### Description Text\n\nA description in natural language that defines the expected quality of the data.\nThis is useful to express requirements or expectation when discussing the data contract with stakeholders.\nLater in the development process, these might be translated into an executable check (such as `sql`).\nIt can also be used as a prompt to check the data with an AI engine.\n\n| Field | Type | Description |\n|-------------|----------|--------------------------------------------------------------------|\n| type | `string` | `text` |\n| description | `string` | A plain text describing the quality attribute in natural language. |\n\nExample:\n\n```yaml\nmodels:\n my_table:\n fields:\n account_iban:\n quality:\n - type: text\n description: Must be a valid IBAN. Must not be empty.\n```\n\n#### SQL\n\nAn individual SQL query that returns a single number that can be compared with a threshold. The SQL query must be in the SQL dialect of the provided server.\n\n> __Note:__ Establish a secure development process and use read-only connections, as the misuse of SQL queries can lead to SQL injection attacks.\n\n\n| Field | Type | Description |\n|----------------------------|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `sql` |\n| description | `string` | A plain text describing the quality of the data. |\n| query | `string` | A SQL query that returns a single number to compare with the threshold. |\n| dialect | `string` | The SQL dialect that is used for the query. Should be compatible to the server type. Examples: `postgres`, `spark`, `bigquery`, `snowflake`, `duckdb`, ... |\n| mustBe | `integer` | The threshold to check the return value of the query |\n| mustNotBe | `integer` | The threshold to check the return value of the query |\n| mustBeGreaterThan | `integer` | The threshold to check the return value of the query |\n| mustBeGreaterThanOrEqualTo | `integer` | The threshold to check the return value of the query |\n| mustBeLessThan | `integer` | The threshold to check the return value of the query |\n| mustBeLessThanOrEqualTo | `integer` | The threshold to check the return value of the query |\n| mustBeBetween | array of two integers | The threshold to check the return value of the query. Boundaries are inclusive. |\n| mustNotBeBetween | array of two integers | The threshold to check the return value of the query. Boundaries are inclusive. |\n\nIn the query the following placeholders can be used:\n\n| Placeholder | Description |\n|-------------|----------------------------------------------------------------------------------------|\n| `{model}` | The name of the model that is checked. |\n| `{table}` | Alias for `{model}`. |\n| `{field}` | The name of the field that is checked (only if the quality is defined on field-level). |\n| `{column}` | Alias for `{field}`. |\n\nExample:\n\n```yaml\nmodels:\n orders:\n quality:\n - type: sql\n description: The maximum duration between two orders must be less that 3600 seconds\n query: |\n SELECT MAX(duration) AS max_duration \n FROM (\n SELECT EXTRACT(EPOCH FROM (order_timestamp - LAG(order_timestamp) OVER (ORDER BY order_timestamp))) AS duration \n FROM {model}\n )\n mustBeLessThan: 3600\n```\n\nSQL queries allow powerful checks for custom business logic.\nA SQL query should run not longer than 10 minutes.\n\n#### Custom\n\nYou can define custom quality attributes that are specific to a data quality engine.\n\n#### Custom (Engine: Soda)\n\nSoda has a number of predefined quality [checks](https://docs.soda.io/soda/data-contracts-checks.html) that can be referenced as quality attributes.\n\nSoda checks can be applied on model and field level.\n\n> Note: Soda Data contract check reference is experimental and may change in the future. Currently only supported by Postgres, Snowflake, and Spark (Databricks)\n\n| Field | Type | Description |\n|---------------|----------|-----------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | `custom` |\n| description | `string` | Optional. A plain text describing the quality attribute in natural language. |\n| engine | `string` | `soda` |\n| implementation | `object` | A check type as defined in the [Data contract check reference](https://docs.soda.io/soda/data-contracts-checks.html) |\n\n\nSee the [Data contract check reference](https://docs.soda.io/soda/data-contracts-checks.html) for all possible types and configuration values.\n\n\nExample:\n\n```yaml\nmodels:\n my_table:\n fields:\n order_id:\n type: string\n quality:\n - type: custom\n description: This is a check on field level\n engine: soda\n implementation:\n type: no_duplicate_values\n carrier:\n type: string\n shipment_numer:\n type: string\n quality:\n - type: custom\n description: This is a check on model level\n engine: soda\n implementation:\n type: duplicate_percent\n columns:\n - carrier\n - shipment_numer\n must_be_less_than: 1.0\n - type: custom\n description: This is a check on model level\n engine: soda\n implementation:\n type: row_count\n must_be_greater_than: 500000\n```\n\n#### Custom (Engine: Great Expectations)\n\nQuality attributes defined as Great Expectations [Expectation](https://greatexpectations.io/expectations/).\n\nExpectations are applied on model level.\n\n| Field | Type | Description |\n|---------------|----------|-----------------------------------------------------------------------------------------------------|\n| description | `string` | Optional. A plain text describing the quality attribute in natural language. |\n| engine | `string` | `great-expectations` |\n| implementation | `object` | An expectation type as listed in [Expectation](https://greatexpectations.io/expectations/) as YAML. |\n\nExample:\n\n```yaml\nmodels:\n my_table:\n quality:\n - type: custom\n engine: great-expectations\n implementation:\n expectation_type: expect_table_row_count_to_be_between\n kwargs:\n min_value: 10000\n max_value: 50000\n meta:\n notes: \"This expectation is crucial to avoid processing datasets that are too small or too large.\"\n - type: custom\n engine: great-expectations\n description: \"Check that passenger_count values are between 1 and 6.\"\n implementation:\n expectation_type: expect_column_values_to_be_between\n kwargs:\n column: passenger_count\n max_value: 6\n min_value: 1\n mostly: 1.0\n strict_max: false\n strict_min: false\n meta:\n tags:\n - business-critical\n - range_check\n```\n\n\n### Lineage Object\n\nField level lineage provides optional fine-grained information where the data comes from and how it was transformed.\n\nThe lineage object is based on the OpenLinage [Column Level Lineage Dataset Facet](https://openlineage.io/docs/spec/facets/dataset-facets/column_lineage_facet) to describe the input fields.\n\n\n\n| Field | Type | Description |\n|-------------|---------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| inputFields | Array of [InputField Object](#inputfield-object) | The input fields refer to specific fields, columns, or data points from source systems or other data contracts that feed into a particular transformation, calculation, or final result. |\n\n\n#### InputField Object\n\n| Field | Type | Description |\n|-----------------|----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| namespace | `string` | The input dataset namespace, such as the name of the source system or the domain of another data contract. Examples: `com.example.crm`, `checkout`, snowflake://{account name}. [More on namespace](https://openlineage.io/blog/whats-in-a-namespace/#namespaces-in-the-spec) |\n| name | `string` | The input dataset name, such as a reference to a data contract, a fully qualified table name, a Kafka topic. |\n| field | `string` | The input field name, such as the field in an upstream data contract, a table column or a JSON Path. |\n| transformations | Array of [Transformation Object](#transformation-object) | Optional. This describes how the input field data was used to generate the final result. |\n\n#### Transformation Object\n\n| Field | Type | Description |\n|-------------|-----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| type | `string` | Indicates how direct is the relationship e.g. in query. Allows values are: `DIRECT` and `INDIRECT`. |\n| subtype | `string` | Optional. Contains more specific information about the transformation.
Allowed values for type `DIRECT`: `IDENTITY`, `TRANSFORMATION`, `AGGREGATION`.
Allowed values for type `INDIRECT`: `JOIN`, `GROUP_BY`, `FILTER`, `SORT`, `WINDOW`, `CONDITIONAL`. |\n| description | `string` | Optional. A string representation of the transformation applied. |\n| masking | `boolean` | Optional. Boolean value indicating if the input value was obfuscated during the transformation. |\n\n\nExample:\n\n```yaml\nmodels:\n orders:\n fields:\n order_id:\n type: string\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: order_id\n transformations:\n - type: DIRECT\n subtype: IDENTITY\n description: The order ID from the checkout order\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: order_timestamp\n transformations:\n - type: INDIRECT\n subtype: SORT\n customer_email_address_hash:\n type: string\n lineage:\n inputFields:\n - namespace: com.example.service.checkout\n name: checkout_db.orders\n field: email_address\n transformations:\n - type: DIRECT\n subtype: Transformation\n description: The email address from the checkout order, hashed with SHA-256\n masking: true\n\n```\n\n\n\n### Config Object\n\nThe config field can be used to set additional metadata that may be used by tools, e.g. to define a namespace for code generation, specify physical data types, toggle tests, etc.\n\nA config field can be added with any name. The value can be null, a primitive, an array or an object.\n\nFor developer experience, a list of well-known field names is maintained here, as these fields are used in the Data Contract CLI:\n\n\n| Field | Type | Description |\n|-----------------|----------|----------------------------------------------------------------------------------------------------------------|\n| avroNamespace | `string` | (Only on model level) The namespace to use when importing and exporting the data model from / to Apache Avro. |\n| avroType | `string` | (Only on field level) Specify the field type to use when exporting the data model to Apache Avro. |\n| avroLogicalType | `string` | (Only on field level) Specify the logical field type to use when exporting the data model to Apache Avro. |\n| bigqueryType | `string` | (Only on field level) Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)` |\n| snowflakeType | `string` | (Only on field level) Specify the physical column type that is used in a Snowflake table, e.g, `TIMESTAMP_LTZ` |\n| redshiftType | `string` | (Only on field level) Specify the physical column type that is used in a Redshift table, e.g, `SMALLINT` |\n| sqlserverType | `string` | (Only on field level) Specify the physical column type that is used in a SQL Server table, e.g, `DATETIME2` |\n| databricksType | `string` | (Only on field level) Specify the physical column type that is used in a Databricks table |\n| glueType | `string` | (Only on field level) Specify the physical column type that is used in a AWS Glue Data Catalog table |\n\nThis object _MAY_ be extended with [Specification Extensions](#specification-extensions).\n\nExample:\n\n```\nmodels:\n orders:\n config:\n avroNamespace: \"my.namespace\"\n fields:\n my_field_1:\n description: Example for AVRO with Timestamp (millisecond precision)\n type: timestamp\n config:\n avroType: long\n avroLogicalType: timestamp-millis\n snowflakeType: timestamp_tz\n```\n\n\n### Data Types\n\nThe following data types are supported for model fields and definitions:\n\n- Unicode character sequence: `string`, `text`, `varchar`\n- Any numeric type, either integers or floating point numbers: `number`, `decimal`, `numeric`\n- 32-bit signed integer: `int`, `integer`\n- 64-bit signed integer: `long`, `bigint`\n- Single precision (32-bit) IEEE 754 floating-point number: `float`\n- Double precision (64-bit) IEEE 754 floating-point number: `double`\n- Binary value: `boolean`\n- Timestamp with timezone: `timestamp`, `timestamp_tz`\n- Timestamp with no timezone: `timestamp_ntz`\n- Date with no time information: `date`\n- Array: `array`\n- Map: `map` (may not be supported by some server types)\n- Sequence of 8-bit unsigned bytes: `bytes`\n- Complex type: `object`, `record`, `struct`\n- No value: `null`\n\n### Specification Extensions\n\nWhile the Data Contract Specification tries to accommodate most use cases, additional data can be added to extend the specification at certain points.\n\nA custom field can be added with any name. The value can be null, a primitive, an array or an object.\n\n\nTooling\n---\n- [Data Contract CLI](https://github.com/datacontract/datacontract-cli) is an open-source CLI tool to help you create, develop, and maintain your data contracts.\n- [Data Contract Manager](https://www.datamesh-manager.com/) is a commercial tool to manage data contracts. It includes a data contract catalog, a Web-Editor, and a request and approval workflow to automate access to data products for a full enterprise data marketplace.\n- [Data Contract GPT](https://gpt.datacontract.com) is a custom GPT that can help you write data contracts.\n- [Data Contract Editor](https://editor.datacontract.com) is an open-source editor for Data Contracts, including a live html preview.\n\nCode Completion\n---\nThe [JSON Schema](https://datacontract.com/datacontract.schema.json) of the current data contract specification is registered in [Schema Store](https://www.schemastore.org/), which brings code completion and syntax checks for all major IDEs.\nIntelliJ comes with a built-in YAML plugin which will show you autocompletions.\nFor VS Code we recommend to install the [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) plugin.\nNo additional configuration is required.\n\nAutocompletion is then enabled for files following these patterns:\n\n```\ndatacontract.yaml\ndatacontract.yml\n*-datacontract.yaml\n*-datacontract.yml\n*.datacontract.yaml\n*.datacontract.yml\ndatacontract-*.yaml\ndatacontract-*.yml\n**/datacontract/*.yml\n**/datacontract/*.yaml\n**/datacontracts/*.yml\n**/datacontracts/*.yaml\n```\n\n\nAuthors\n---\nThe Data Contract Specification was originally created by [Jochen Christ](https://www.linkedin.com/in/jochenchrist/) and [Dr. Simon Harrer](https://www.linkedin.com/in/simonharrer/), and is currently maintained by them.\n\n\nContributing\n---\nContributions are welcome! Please open an issue or a pull request.\n\nLicense\n---\n[MIT License](LICENSE)\n\n\n" }, { "path": "versions/1.1.0/datacontract.init.yaml", "content": "dataContractSpecification: 1.1.0\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# production:\n# type: s3\n# location: s3://\n# format: parquet\n# delimiter: new_line\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### models\n\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type:\n# description:\n\n\n### definitions\n\n# definitions:\n# my_field:\n# domain:\n# name:\n# title:\n# type:\n# description:\n# example:\n# pii:\n# classification:\n\n\n### servicelevels\n\n#servicelevels:\n# availability:\n# description: The server is available during support hours\n# percentage: 99.9%\n# retention:\n# description: Data is retained for one year because!\n# period: P1Y\n# unlimited: false\n# latency:\n# description: Data is available within 25 hours after the order was placed\n# threshold: 25h\n# sourceTimestampField: orders.order_timestamp\n# processedTimestampField: orders.processed_timestamp\n# freshness:\n# description: The age of the youngest row in a table.\n# threshold: 25h\n# timestampField: orders.order_timestamp\n# frequency:\n# description: Data is delivered once a day\n# type: batch # or streaming\n# interval: daily # for batch, either or cron\n# cron: 0 0 * * * # for batch, either or interval\n# support:\n# description: The data is available during typical business hours at headquarters\n# time: 9am to 5pm in EST on business days\n# responseTime: 1h\n# backup:\n# description: Data is backed up once a week, every Sunday at 0:00 UTC.\n# interval: weekly\n# cron: 0 0 * * 0\n# recoveryTime: 24 hours\n# recoveryPoint: 1 week" }, { "path": "versions/1.1.0/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"title\": \"DataContractSpecification\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"title\": \"DataContractSpecificationVersion\",\n \"enum\": [\n \"1.1.0\",\n \"0.9.3\",\n \"0.9.2\",\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"status\": {\n \"type\": \"string\",\n \"description\": \"The status of the data contract. Can be proposed, in development, active, retired.\",\n \"examples\": [\n \"proposed\",\n \"in development\",\n \"active\",\n \"deprecated\",\n \"retired\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\",\n \"additionalProperties\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"description\": \"Information about the servers.\",\n \"additionalProperties\": {\n \"$ref\": \"#/$defs/BaseServer\",\n \"allOf\": [\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"bigquery\"\n }\n }\n },\n \"then\": {\n \"$ref\": \"#/$defs/BigQueryServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"s3\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/S3Server\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sftp\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SftpServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"redshift\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/RedshiftServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"azure\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/AzureServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sqlserver\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SqlserverServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"snowflake\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SnowflakeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"databricks\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DatabricksServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"dataframe\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DataframeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"glue\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/GlueServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"oracle\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/OracleServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kafka\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KafkaServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"pubsub\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PubSubServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kinesis\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KinesisDataStreamsServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"trino\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/TrinoServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"local\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/LocalServer\"\n }\n }\n ]\n }\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"policies\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the policy.\",\n \"examples\": [\n \"privacy\",\n \"security\",\n \"retention\",\n \"compliance\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the policy.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"A URL to the policy document.\"\n }\n },\n \"additionalProperties\": true\n },\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n },\n \"additionalProperties\": true\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Model\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, view, object. Default: table.\",\n \"type\": \"string\",\n \"title\": \"ModelType\",\n \"default\": \"table\",\n \"enum\": [\n \"table\",\n \"view\",\n \"object\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the model. Especially useful if the model name is cryptic or contains abbreviations.\",\n \"examples\": [\n \"Purchase Orders\",\n \"Air Shipments\"\n ]\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Field\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"required\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if this field must contain a value and may not be null.\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"primary\": {\n \"type\": \"boolean\",\n \"deprecationMessage\": \"Use the primaryKey field instead.\"\n },\n \"primaryKey\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"If this field is a primary key.\"\n },\n \"references\": {\n \"type\": \"string\",\n \"description\": \"The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship.\",\n \"examples\": [\n \"orders.order_id\",\n \"model.nested_field.field\"\n ]\n },\n \"unique\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if the value must be unique within the model.\"\n },\n \"enum\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"uniqueItems\": true,\n \"description\": \"A value must be equal to one of the elements in this array value. Only evaluated if the value is not null.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must greater than, or equal to, the value of this. Only applies to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must less than, or equal to, the value of this. Only applies to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"A specific format the value must comply with (e.g., 'email', 'uri', 'uuid').\",\n \"examples\": [\n \"email\",\n \"uri\",\n \"uuid\"\n ]\n },\n \"precision\": {\n \"type\": \"number\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"number\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression the value must match. Only applies to string types.\",\n \"examples\": [\n \"^[a-zA-Z0-9_-]+$\"\n ]\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\n \"sensitive\",\n \"restricted\",\n \"internal\",\n \"public\"\n ]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"lineage\": {\n \"$ref\": \"#/$defs/Lineage\"\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for field configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroType\": {\n \"type\": \"string\",\n \"description\": \"Specify the field type to use when exporting the data model to Apache Avro.\"\n },\n \"avroLogicalType\": {\n \"type\": \"string\",\n \"description\": \"Specify the logical field type to use when exporting the data model to Apache Avro.\"\n },\n \"bigqueryType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)`.\"\n },\n \"snowflakeType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Snowflake table, e.g., `TIMESTAMP_LTZ`.\"\n },\n \"redshiftType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Redshift table, e.g., `SMALLINT`.\"\n },\n \"sqlserverType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a SQL Server table, e.g., `DATETIME2`.\"\n },\n \"databricksType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Databricks Unity Catalog table.\"\n },\n \"glueType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in an AWS Glue Data Catalog table.\"\n }\n }\n }\n }\n }\n },\n \"primaryKey\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"The compound primary key of the model.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"examples\": {\n \"type\": \"array\"\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for model configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroNamespace\": {\n \"type\": \"string\",\n \"description\": \"The namespace to use when importing and exporting the data model from / to Apache Avro.\"\n }\n }\n }\n }\n }\n },\n \"definitions\": {\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"type\": \"object\",\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9/_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Definition\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\",\n \"deprecationMessage\": \"This field is deprecated. Encode the domain into the ID using slashes.\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\",\n \"deprecationMessage\": \"This field is deprecated. Encode the name into the ID using slashes.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"Example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n }\n },\n \"servicelevels\": {\n \"type\": \"object\",\n \"description\": \"Specifies the service level agreements for the provided data, including availability, data retention policies, latency requirements, data freshness, update frequency, support availability, and backup policies.\",\n \"properties\": {\n \"availability\": {\n \"type\": \"object\",\n \"description\": \"Availability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the availability service level.\",\n \"example\": \"The server is available during support hours\"\n },\n \"percentage\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the guaranteed uptime in percent (e.g., `99.9%`)\",\n \"pattern\": \"^\\\\d+(\\\\.\\\\d+)?%$\",\n \"example\": \"99.9%\"\n }\n }\n },\n \"retention\": {\n \"type\": \"object\",\n \"description\": \"Retention covers the period how long data will be available.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the retention service level.\",\n \"example\": \"Data is retained for one year.\"\n },\n \"period\": {\n \"type\": \"string\",\n \"description\": \"An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`).\",\n \"example\": \"P1Y\"\n },\n \"unlimited\": {\n \"type\": \"boolean\",\n \"description\": \"An optional indicator that data is kept forever.\",\n \"example\": false\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the period refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"latency\": {\n \"type\": \"object\",\n \"description\": \"Latency refers to the maximum amount of time from the source to its destination.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the latency service level.\",\n \"example\": \"Data is available within 25 hours after the order was placed.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`).\",\n \"example\": \"25h\"\n },\n \"sourceTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp when the data was provided at the source.\",\n \"example\": \"orders.order_timestamp\"\n },\n \"processedTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract.\",\n \"example\": \"orders.processed_timestamp\"\n }\n }\n },\n \"freshness\": {\n \"type\": \"object\",\n \"description\": \"The maximum age of the youngest row in a table.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the freshness service level.\",\n \"example\": \"The age of the youngest row in a table is within 25 hours.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g., `PT24H`).\",\n \"example\": \"25h\"\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the threshold refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"frequency\": {\n \"type\": \"object\",\n \"description\": \"Frequency describes how often data is updated.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the frequency service level.\",\n \"example\": \"Data is delivered once a day.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"batch\",\n \"micro-batching\",\n \"streaming\",\n \"manual\"\n ],\n \"description\": \"The method of data processing.\",\n \"example\": \"batch\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`.\",\n \"example\": \"daily\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * *\"\n }\n }\n },\n \"support\": {\n \"type\": \"object\",\n \"description\": \"Support describes the times when support will be available for contact.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the support service level.\",\n \"example\": \"The data is available during typical business hours at headquarters.\"\n },\n \"time\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`.\",\n \"example\": \"9am to 5pm in EST on business days\"\n },\n \"responseTime\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with.\",\n \"example\": \"24 hours\"\n }\n }\n },\n \"backup\": {\n \"type\": \"object\",\n \"description\": \"Backup specifies details about data backup procedures.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the backup service level.\",\n \"example\": \"Data is backed up once a week, every Sunday at 0:00 UTC.\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"An optional interval that defines how often data will be backed up, e.g., `daily`.\",\n \"example\": \"weekly\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"An optional cron expression when data will be backed up, e.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * 0\"\n },\n \"recoveryTime\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours).\",\n \"example\": \"24 hours\"\n },\n \"recoveryPoint\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours).\",\n \"example\": \"1 week\"\n }\n }\n }\n }\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"description\": \"Tags to facilitate searching and filtering.\",\n \"examples\": [\n \"databricks\",\n \"pii\",\n \"sensitive\"\n ]\n },\n \"description\": \"Tags to facilitate searching and filtering.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ],\n \"$defs\": {\n \"FieldType\": {\n \"type\": \"string\",\n \"title\": \"FieldType\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\",\n \"decimal\",\n \"numeric\",\n \"int\",\n \"integer\",\n \"long\",\n \"bigint\",\n \"float\",\n \"double\",\n \"string\",\n \"text\",\n \"varchar\",\n \"boolean\",\n \"timestamp\",\n \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"array\",\n \"map\",\n \"object\",\n \"record\",\n \"struct\",\n \"bytes\",\n \"null\"\n ]\n },\n \"BaseServer\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the servers.\"\n },\n \"environment\": {\n \"type\": \"string\",\n \"description\": \"The environment in which the servers are running. Examples: prod, sit, stg.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the data product technology that implements the data contract.\",\n \"enum\": [\n \"bigquery\",\n \"BigQuery\",\n \"s3\",\n \"sftp\",\n \"redshift\",\n \"azure\",\n \"sqlserver\",\n \"snowflake\",\n \"databricks\",\n \"dataframe\",\n \"glue\",\n \"postgres\",\n \"oracle\",\n \"kafka\",\n \"pubsub\",\n \"kinesis\",\n \"trino\",\n \"local\"\n ]\n },\n \"roles\": {\n \"description\": \" An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The name of the role.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the role and what access the role provides.\"\n }\n },\n \"required\": [\n \"name\"\n ]\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\"\n ]\n },\n \"BigQueryServer\": {\n \"type\": \"object\",\n \"title\": \"BigQueryServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"The GCP dataset name.\"\n }\n },\n \"required\": [\n \"project\",\n \"dataset\"\n ]\n },\n \"S3Server\": {\n \"type\": \"object\",\n \"title\": \"S3Server\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"S3 URL, starting with `s3://`\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}/*.json\"\n ]\n },\n \"endpointUrl\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The server endpoint for S3-compatible servers.\",\n \"examples\": [\n \"https://minio.example.com\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"SftpServer\": {\n \"type\": \"object\",\n \"title\": \"SftpServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"pattern\": \"^sftp://.*\",\n \"description\": \"SFTP URL, starting with `sftp://`\",\n \"examples\": [\n \"sftp://123.123.12.123/{model}/*.json\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"RedshiftServer\": {\n \"type\": \"object\",\n \"title\": \"RedshiftServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the host name.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"clusterIdentifier\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's identifier.\",\n \"examples\": [\n \"redshift-prod-eu\",\n \"analytics-cluster\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"An optional string describing the cluster's port.\",\n \"examples\": [\n 5439\n ]\n },\n \"endpoint\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's endpoint.\",\n \"examples\": [\n \"analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"AzureServer\": {\n \"type\": \"object\",\n \"title\": \"AzureServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"Path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs. Recommended pattern is 'abfss:///'\",\n \"examples\": [\n \"abfss://my_container_name/path\",\n \"abfss://my_container_name/path/*.json\",\n \"az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet\",\n \"abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\",\n \"format\"\n ]\n },\n \"SqlserverServer\": {\n \"type\": \"object\",\n \"title\": \"SqlserverServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\",\n \"default\": 1433,\n \"examples\": [\n 1433\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"database\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"dbo\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"database\",\n \"schema\"\n ]\n },\n \"SnowflakeServer\": {\n \"type\": \"object\",\n \"title\": \"SnowflakeServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"DatabricksServer\": {\n \"type\": \"object\",\n \"title\": \"DatabricksServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Databricks host\",\n \"examples\": [\n \"dbc-abcdefgh-1234.cloud.databricks.com\"\n ]\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the Hive or Unity catalog\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The schema name in the catalog\"\n }\n },\n \"required\": [\n \"catalog\",\n \"schema\"\n ]\n },\n \"DataframeServer\": {\n \"type\": \"object\",\n \"title\": \"DataframeServer\",\n \"required\": [\n \"type\"\n ]\n },\n \"GlueServer\": {\n \"type\": \"object\",\n \"title\": \"GlueServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue account\",\n \"examples\": [\n \"1234-5678-9012\"\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue database name\",\n \"examples\": [\n \"my_database\"\n ]\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The AWS S3 path. Must be in the form of a URL.\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the files\",\n \"examples\": [\n \"parquet\",\n \"csv\",\n \"json\",\n \"delta\"\n ]\n }\n },\n \"required\": [\n \"account\",\n \"database\"\n ]\n },\n \"PostgresServer\": {\n \"type\": \"object\",\n \"title\": \"PostgresServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"public\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\",\n \"schema\"\n ]\n },\n \"OracleServer\": {\n \"type\": \"object\",\n \"title\": \"OracleServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the oracle server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the oracle server.\",\n \"examples\": [\n 1523\n ]\n },\n \"serviceName\": {\n \"type\": \"string\",\n \"description\": \"The name of the service.\",\n \"examples\": [\n \"service\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"serviceName\"\n ]\n },\n \"KafkaServer\": {\n \"type\": \"object\",\n \"title\": \"KafkaServer\",\n \"description\": \"Kafka Server\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The bootstrap server of the kafka cluster.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the message. Examples: json, avro, protobuf.\",\n \"default\": \"json\"\n }\n },\n \"required\": [\n \"host\",\n \"topic\"\n ]\n },\n \"PubSubServer\": {\n \"type\": \"object\",\n \"title\": \"PubSubServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n }\n },\n \"required\": [\n \"project\",\n \"topic\"\n ]\n },\n \"KinesisDataStreamsServer\": {\n \"type\": \"object\",\n \"title\": \"KinesisDataStreamsServer\",\n \"description\": \"Kinesis Data Streams Server\",\n \"properties\": {\n \"stream\": {\n \"type\": \"string\",\n \"description\": \"The name of the Kinesis data stream.\"\n },\n \"region\": {\n \"type\": \"string\",\n \"description\": \"AWS region.\",\n \"examples\": [\n \"eu-west-1\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the record\",\n \"examples\": [\n \"json\",\n \"avro\",\n \"protobuf\"\n ]\n }\n },\n \"required\": [\n \"stream\"\n ]\n },\n \"TrinoServer\": {\n \"type\": \"object\",\n \"title\": \"TrinoServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Trino host URL.\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The Trino port.\"\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the catalog.\",\n \"examples\": [\n \"hive\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"my_schema\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"catalog\",\n \"schema\"\n ]\n },\n \"LocalServer\": {\n \"type\": \"object\",\n \"title\": \"LocalServer\",\n \"properties\": {\n \"path\": {\n \"type\": \"string\",\n \"description\": \"The relative or absolute path to the data file(s).\",\n \"examples\": [\n \"./folder/data.parquet\",\n \"./folder/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the file(s)\",\n \"examples\": [\n \"json\",\n \"parquet\",\n \"delta\",\n \"csv\"\n ]\n }\n },\n \"required\": [\n \"path\",\n \"format\"\n ]\n },\n \"Quality\": {\n \"allOf\": [\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of quality check\",\n \"enum\": [\n \"text\",\n \"library\",\n \"sql\",\n \"custom\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n }\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"text\"\n }\n }\n },\n \"then\": {\n \"required\": [\n \"description\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sql\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"query\": {\n \"type\": \"string\",\n \"description\": \"A SQL query that returns a single number to compare with the threshold.\"\n },\n \"dialect\": {\n \"type\": \"string\",\n \"description\": \"The SQL dialect that is used for the query. Should be compatible to the server.type.\",\n \"examples\": [\n \"athena\",\n \"bigquery\",\n \"redshift\",\n \"snowflake\",\n \"trino\",\n \"postgres\",\n \"oracle\"\n ]\n },\n \"mustBe\": {\n \"type\": \"number\"\n },\n \"mustNotBe\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterThanOrEqualTo\": {\n \"type\": \"number\"\n },\n \"mustBeLessThan\": {\n \"type\": \"number\"\n },\n \"mustBeLessThanOrEqualTo\": {\n \"type\": \"number\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n }\n },\n \"required\": [\n \"query\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"library\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"rule\": {\n \"type\": \"string\",\n \"description\": \"Define a data quality check based on the predefined rules as per ODCS.\",\n \"examples\": [\"duplicateCount\", \"validValues\", \"rowCount\"]\n },\n \"mustBe\": {\n \"description\": \"Must be equal to the value to be valid. When using numbers, it is equivalent to '='.\"\n },\n \"mustNotBe\": {\n \"description\": \"Must not be equal to the value to be valid. When using numbers, it is equivalent to '!='.\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than the value to be valid. It is equivalent to '>'.\"\n },\n \"mustBeGreaterOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than or equal to the value to be valid. It is equivalent to '>='.\"\n },\n \"mustBeLessThan\": {\n \"type\": \"number\",\n \"description\": \"Must be less than the value to be valid. It is equivalent to '<'.\"\n },\n \"mustBeLessOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be less than or equal to the value to be valid. It is equivalent to '<='.\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must not be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n }\n },\n \"required\": [\n \"rule\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"custom\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n },\n \"engine\": {\n \"type\": \"string\",\n \"examples\": [\n \"soda\",\n \"great-expectations\"\n ],\n \"description\": \"The engine used for custom quality checks.\"\n },\n \"implementation\": {\n \"type\": [\n \"object\",\n \"array\",\n \"string\"\n ],\n \"description\": \"Engine-specific quality checks and expectations.\"\n }\n },\n \"required\": [\n \"engine\"\n ]\n }\n }\n ]\n },\n \"Lineage\": {\n \"type\": \"object\",\n \"properties\": {\n \"inputFields\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"namespace\": {\n \"type\": \"string\",\n \"description\": \"The input dataset namespace\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The input dataset name\"\n },\n \"field\": {\n \"type\": \"string\",\n \"description\": \"The input field\"\n },\n \"transformations\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"description\": \"The type of the transformation. Allowed values are: DIRECT, INDIRECT\",\n \"type\": \"string\"\n },\n \"subtype\": {\n \"type\": \"string\",\n \"description\": \"The subtype of the transformation\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\"\n },\n \"masking\": {\n \"type\": \"boolean\",\n \"description\": \"is transformation masking the data or not\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"additionalProperties\": true\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"namespace\",\n \"name\",\n \"field\"\n ]\n }\n },\n \"transformationDescription\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\",\n \"deprecated\": true\n },\n \"transformationType\": {\n \"type\": \"string\",\n \"description\": \"IDENTITY|MASKED reflects a clearly defined behavior. IDENTITY: exact same as input; MASKED: no original data available (like a hash of PII for example)\",\n \"deprecated\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"inputFields\"\n ]\n }\n }\n}" }, { "path": "versions/1.1.0/definition.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"properties\": {\n \"id\": {\n \"type\": \"string\",\n \"description\": \"A unique identifier for this definition. Encode the domain into the ID, separated by slashes.\",\n \"examples\": [\n \"checkout/order_id\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n}\n" }, { "path": "versions/1.2.0/datacontract.init.yaml", "content": "dataContractSpecification: 1.2.0\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# production:\n# type: s3\n# location: s3://\n# format: parquet\n# delimiter: new_line\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### models\n\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type:\n# description:\n\n\n### definitions\n\n# definitions:\n# my_field:\n# domain:\n# name:\n# title:\n# type:\n# description:\n# example:\n# pii:\n# classification:\n\n\n### servicelevels\n\n#servicelevels:\n# availability:\n# description: The server is available during support hours\n# percentage: 99.9%\n# retention:\n# description: Data is retained for one year because!\n# period: P1Y\n# unlimited: false\n# latency:\n# description: Data is available within 25 hours after the order was placed\n# threshold: 25h\n# sourceTimestampField: orders.order_timestamp\n# processedTimestampField: orders.processed_timestamp\n# freshness:\n# description: The age of the youngest row in a table.\n# threshold: 25h\n# timestampField: orders.order_timestamp\n# frequency:\n# description: Data is delivered once a day\n# type: batch # or streaming\n# interval: daily # for batch, either or cron\n# cron: 0 0 * * * # for batch, either or interval\n# support:\n# description: The data is available during typical business hours at headquarters\n# time: 9am to 5pm in EST on business days\n# responseTime: 1h\n# backup:\n# description: Data is backed up once a week, every Sunday at 0:00 UTC.\n# interval: weekly\n# cron: 0 0 * * 0\n# recoveryTime: 24 hours\n# recoveryPoint: 1 week\n" }, { "path": "versions/1.2.0/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"title\": \"DataContractSpecification\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"title\": \"DataContractSpecificationVersion\",\n \"enum\": [\n \"1.2.0\",\n \"1.1.0\",\n \"0.9.3\",\n \"0.9.2\",\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"status\": {\n \"type\": \"string\",\n \"description\": \"The status of the data contract. Can be proposed, in development, active, retired.\",\n \"examples\": [\n \"proposed\",\n \"in development\",\n \"active\",\n \"deprecated\",\n \"retired\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\",\n \"additionalProperties\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"description\": \"Information about the servers.\",\n \"additionalProperties\": {\n \"$ref\": \"#/$defs/BaseServer\",\n \"allOf\": [\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"bigquery\"\n }\n }\n },\n \"then\": {\n \"$ref\": \"#/$defs/BigQueryServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"s3\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/S3Server\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sftp\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SftpServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"redshift\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/RedshiftServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"azure\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/AzureServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sqlserver\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SqlserverServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"snowflake\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SnowflakeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"databricks\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DatabricksServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"dataframe\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DataframeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"glue\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/GlueServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"oracle\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/OracleServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kafka\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KafkaServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"pubsub\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PubSubServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kinesis\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KinesisDataStreamsServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"trino\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/TrinoServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"clickhouse\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/ClickhouseServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"local\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/LocalServer\"\n }\n }\n ]\n }\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"policies\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the policy.\",\n \"examples\": [\n \"privacy\",\n \"security\",\n \"retention\",\n \"compliance\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the policy.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"A URL to the policy document.\"\n }\n },\n \"additionalProperties\": true\n },\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n },\n \"additionalProperties\": true\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Model\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, view, object. Default: table.\",\n \"type\": \"string\",\n \"title\": \"ModelType\",\n \"default\": \"table\",\n \"enum\": [\n \"table\",\n \"view\",\n \"object\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the model. Especially useful if the model name is cryptic or contains abbreviations.\",\n \"examples\": [\n \"Purchase Orders\",\n \"Air Shipments\"\n ]\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Field\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"required\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if this field must contain a value and may not be null.\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"primary\": {\n \"type\": \"boolean\",\n \"deprecationMessage\": \"Use the primaryKey field instead.\"\n },\n \"primaryKey\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"If this field is a primary key.\"\n },\n \"references\": {\n \"type\": \"string\",\n \"description\": \"The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship.\",\n \"examples\": [\n \"orders.order_id\",\n \"model.nested_field.field\"\n ]\n },\n \"unique\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if the value must be unique within the model.\"\n },\n \"enum\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"uniqueItems\": true,\n \"description\": \"A value must be equal to one of the elements in this array value. Only evaluated if the value is not null.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must greater than, or equal to, the value of this. Only applies to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must less than, or equal to, the value of this. Only applies to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"A specific format the value must comply with (e.g., 'email', 'uri', 'uuid').\",\n \"examples\": [\n \"email\",\n \"uri\",\n \"uuid\"\n ]\n },\n \"precision\": {\n \"type\": \"number\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"number\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression the value must match. Only applies to string types.\",\n \"examples\": [\n \"^[a-zA-Z0-9_-]+$\"\n ]\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\n \"sensitive\",\n \"restricted\",\n \"internal\",\n \"public\"\n ]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"lineage\": {\n \"$ref\": \"#/$defs/Lineage\"\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for field configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroType\": {\n \"type\": \"string\",\n \"description\": \"Specify the field type to use when exporting the data model to Apache Avro.\"\n },\n \"avroLogicalType\": {\n \"type\": \"string\",\n \"description\": \"Specify the logical field type to use when exporting the data model to Apache Avro.\"\n },\n \"bigqueryType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)`.\"\n },\n \"snowflakeType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Snowflake table, e.g., `TIMESTAMP_LTZ`.\"\n },\n \"redshiftType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Redshift table, e.g., `SMALLINT`.\"\n },\n \"sqlserverType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a SQL Server table, e.g., `DATETIME2`.\"\n },\n \"databricksType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Databricks Unity Catalog table.\"\n },\n \"glueType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in an AWS Glue Data Catalog table.\"\n }\n }\n }\n }\n }\n },\n \"primaryKey\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"The compound primary key of the model.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"examples\": {\n \"type\": \"array\"\n },\n \"additionalFields\": {\n \"type\": \"boolean\",\n \"description\": \" Specify, if the model can have additional fields that are not defined in the contract. \",\n \"default\": false\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for model configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroNamespace\": {\n \"type\": \"string\",\n \"description\": \"The namespace to use when importing and exporting the data model from / to Apache Avro.\"\n }\n }\n }\n }\n }\n },\n \"definitions\": {\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"type\": \"object\",\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9/_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Definition\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\",\n \"deprecationMessage\": \"This field is deprecated. Encode the domain into the ID using slashes.\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\",\n \"deprecationMessage\": \"This field is deprecated. Encode the name into the ID using slashes.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"Example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n }\n },\n \"servicelevels\": {\n \"type\": \"object\",\n \"description\": \"Specifies the service level agreements for the provided data, including availability, data retention policies, latency requirements, data freshness, update frequency, support availability, and backup policies.\",\n \"properties\": {\n \"availability\": {\n \"type\": \"object\",\n \"description\": \"Availability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the availability service level.\",\n \"example\": \"The server is available during support hours\"\n },\n \"percentage\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the guaranteed uptime in percent (e.g., `99.9%`)\",\n \"pattern\": \"^\\\\d+(\\\\.\\\\d+)?%$\",\n \"example\": \"99.9%\"\n }\n }\n },\n \"retention\": {\n \"type\": \"object\",\n \"description\": \"Retention covers the period how long data will be available.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the retention service level.\",\n \"example\": \"Data is retained for one year.\"\n },\n \"period\": {\n \"type\": \"string\",\n \"description\": \"An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`).\",\n \"example\": \"P1Y\"\n },\n \"unlimited\": {\n \"type\": \"boolean\",\n \"description\": \"An optional indicator that data is kept forever.\",\n \"example\": false\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the period refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"latency\": {\n \"type\": \"object\",\n \"description\": \"Latency refers to the maximum amount of time from the source to its destination.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the latency service level.\",\n \"example\": \"Data is available within 25 hours after the order was placed.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`).\",\n \"example\": \"25h\"\n },\n \"sourceTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp when the data was provided at the source.\",\n \"example\": \"orders.order_timestamp\"\n },\n \"processedTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract.\",\n \"example\": \"orders.processed_timestamp\"\n }\n }\n },\n \"freshness\": {\n \"type\": \"object\",\n \"description\": \"The maximum age of the youngest row in a table.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the freshness service level.\",\n \"example\": \"The age of the youngest row in a table is within 25 hours.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g., `PT24H`).\",\n \"example\": \"25h\"\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the threshold refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"frequency\": {\n \"type\": \"object\",\n \"description\": \"Frequency describes how often data is updated.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the frequency service level.\",\n \"example\": \"Data is delivered once a day.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"batch\",\n \"micro-batching\",\n \"streaming\",\n \"manual\"\n ],\n \"description\": \"The method of data processing.\",\n \"example\": \"batch\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`.\",\n \"example\": \"daily\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * *\"\n }\n }\n },\n \"support\": {\n \"type\": \"object\",\n \"description\": \"Support describes the times when support will be available for contact.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the support service level.\",\n \"example\": \"The data is available during typical business hours at headquarters.\"\n },\n \"time\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`.\",\n \"example\": \"9am to 5pm in EST on business days\"\n },\n \"responseTime\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with.\",\n \"example\": \"24 hours\"\n }\n }\n },\n \"backup\": {\n \"type\": \"object\",\n \"description\": \"Backup specifies details about data backup procedures.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the backup service level.\",\n \"example\": \"Data is backed up once a week, every Sunday at 0:00 UTC.\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"An optional interval that defines how often data will be backed up, e.g., `daily`.\",\n \"example\": \"weekly\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"An optional cron expression when data will be backed up, e.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * 0\"\n },\n \"recoveryTime\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours).\",\n \"example\": \"24 hours\"\n },\n \"recoveryPoint\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours).\",\n \"example\": \"1 week\"\n }\n }\n }\n }\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"description\": \"Tags to facilitate searching and filtering.\",\n \"examples\": [\n \"databricks\",\n \"pii\",\n \"sensitive\"\n ]\n },\n \"description\": \"Tags to facilitate searching and filtering.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ],\n \"$defs\": {\n \"FieldType\": {\n \"type\": \"string\",\n \"title\": \"FieldType\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\",\n \"decimal\",\n \"numeric\",\n \"int\",\n \"integer\",\n \"long\",\n \"bigint\",\n \"float\",\n \"double\",\n \"string\",\n \"text\",\n \"varchar\",\n \"boolean\",\n \"timestamp\",\n \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"time\",\n \"array\",\n \"map\",\n \"object\",\n \"record\",\n \"struct\",\n \"bytes\",\n \"variant\",\n \"json\",\n \"null\"\n ]\n },\n \"BaseServer\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the servers.\"\n },\n \"environment\": {\n \"type\": \"string\",\n \"description\": \"The environment in which the servers are running. Examples: prod, sit, stg.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the data product technology that implements the data contract.\",\n \"examples\": [\n \"azure\",\n \"bigquery\",\n \"BigQuery\",\n \"clickhouse\",\n \"databricks\",\n \"dataframe\",\n \"glue\",\n \"kafka\",\n \"kinesis\",\n \"local\",\n \"oracle\",\n \"postgres\",\n \"pubsub\",\n \"redshift\",\n \"sftp\",\n \"sqlserver\",\n \"snowflake\",\n \"s3\",\n \"trino\"\n ]\n },\n \"roles\": {\n \"description\": \" An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The name of the role.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the role and what access the role provides.\"\n }\n },\n \"required\": [\n \"name\"\n ]\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\"\n ]\n },\n \"BigQueryServer\": {\n \"type\": \"object\",\n \"title\": \"BigQueryServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"The GCP dataset name.\"\n }\n },\n \"required\": [\n \"project\",\n \"dataset\"\n ]\n },\n \"S3Server\": {\n \"type\": \"object\",\n \"title\": \"S3Server\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"S3 URL, starting with `s3://`\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}/*.json\"\n ]\n },\n \"endpointUrl\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The server endpoint for S3-compatible servers.\",\n \"examples\": [\n \"https://minio.example.com\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"SftpServer\": {\n \"type\": \"object\",\n \"title\": \"SftpServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"pattern\": \"^sftp://.*\",\n \"description\": \"SFTP URL, starting with `sftp://`\",\n \"examples\": [\n \"sftp://123.123.12.123/{model}/*.json\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"RedshiftServer\": {\n \"type\": \"object\",\n \"title\": \"RedshiftServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the host name.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"clusterIdentifier\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's identifier.\",\n \"examples\": [\n \"redshift-prod-eu\",\n \"analytics-cluster\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"An optional string describing the cluster's port.\",\n \"examples\": [\n 5439\n ]\n },\n \"endpoint\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's endpoint.\",\n \"examples\": [\n \"analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"AzureServer\": {\n \"type\": \"object\",\n \"title\": \"AzureServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"Path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs. Recommended pattern is 'abfss:///'\",\n \"examples\": [\n \"abfss://my_container_name/path\",\n \"abfss://my_container_name/path/*.json\",\n \"az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet\",\n \"abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\",\n \"format\"\n ]\n },\n \"SqlserverServer\": {\n \"type\": \"object\",\n \"title\": \"SqlserverServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\",\n \"default\": 1433,\n \"examples\": [\n 1433\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"database\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"dbo\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"database\",\n \"schema\"\n ]\n },\n \"SnowflakeServer\": {\n \"type\": \"object\",\n \"title\": \"SnowflakeServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"DatabricksServer\": {\n \"type\": \"object\",\n \"title\": \"DatabricksServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Databricks host\",\n \"examples\": [\n \"dbc-abcdefgh-1234.cloud.databricks.com\"\n ]\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the Hive or Unity catalog\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The schema name in the catalog\"\n }\n },\n \"required\": [\n \"catalog\",\n \"schema\"\n ]\n },\n \"DataframeServer\": {\n \"type\": \"object\",\n \"title\": \"DataframeServer\",\n \"required\": [\n \"type\"\n ]\n },\n \"GlueServer\": {\n \"type\": \"object\",\n \"title\": \"GlueServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue account\",\n \"examples\": [\n \"1234-5678-9012\"\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue database name\",\n \"examples\": [\n \"my_database\"\n ]\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The AWS S3 path. Must be in the form of a URL.\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the files\",\n \"examples\": [\n \"parquet\",\n \"csv\",\n \"json\",\n \"delta\"\n ]\n }\n },\n \"required\": [\n \"account\",\n \"database\"\n ]\n },\n \"PostgresServer\": {\n \"type\": \"object\",\n \"title\": \"PostgresServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"public\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\",\n \"schema\"\n ]\n },\n \"OracleServer\": {\n \"type\": \"object\",\n \"title\": \"OracleServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the oracle server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the oracle server.\",\n \"examples\": [\n 1523\n ]\n },\n \"serviceName\": {\n \"type\": \"string\",\n \"description\": \"The name of the service.\",\n \"examples\": [\n \"service\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"serviceName\"\n ]\n },\n \"KafkaServer\": {\n \"type\": \"object\",\n \"title\": \"KafkaServer\",\n \"description\": \"Kafka Server\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The bootstrap server of the kafka cluster.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the message. Examples: json, avro, protobuf.\",\n \"default\": \"json\"\n }\n },\n \"required\": [\n \"host\",\n \"topic\"\n ]\n },\n \"PubSubServer\": {\n \"type\": \"object\",\n \"title\": \"PubSubServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n }\n },\n \"required\": [\n \"project\",\n \"topic\"\n ]\n },\n \"KinesisDataStreamsServer\": {\n \"type\": \"object\",\n \"title\": \"KinesisDataStreamsServer\",\n \"description\": \"Kinesis Data Streams Server\",\n \"properties\": {\n \"stream\": {\n \"type\": \"string\",\n \"description\": \"The name of the Kinesis data stream.\"\n },\n \"region\": {\n \"type\": \"string\",\n \"description\": \"AWS region.\",\n \"examples\": [\n \"eu-west-1\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the record\",\n \"examples\": [\n \"json\",\n \"avro\",\n \"protobuf\"\n ]\n }\n },\n \"required\": [\n \"stream\"\n ]\n },\n \"TrinoServer\": {\n \"type\": \"object\",\n \"title\": \"TrinoServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Trino host URL.\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The Trino port.\"\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the catalog.\",\n \"examples\": [\n \"hive\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"my_schema\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"catalog\",\n \"schema\"\n ]\n },\n \"ClickhouseServer\": {\n \"type\": \"object\",\n \"title\": \"ClickhouseServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\"\n ]\n },\n \"LocalServer\": {\n \"type\": \"object\",\n \"title\": \"LocalServer\",\n \"properties\": {\n \"path\": {\n \"type\": \"string\",\n \"description\": \"The relative or absolute path to the data file(s).\",\n \"examples\": [\n \"./folder/data.parquet\",\n \"./folder/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the file(s)\",\n \"examples\": [\n \"json\",\n \"parquet\",\n \"delta\",\n \"csv\"\n ]\n }\n },\n \"required\": [\n \"path\",\n \"format\"\n ]\n },\n \"Quality\": {\n \"allOf\": [\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of quality check\",\n \"enum\": [\n \"text\",\n \"library\",\n \"sql\",\n \"custom\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n }\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"text\"\n }\n }\n },\n \"then\": {\n \"required\": [\n \"description\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sql\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"query\": {\n \"type\": \"string\",\n \"description\": \"A SQL query that returns a single number to compare with the threshold.\"\n },\n \"dialect\": {\n \"type\": \"string\",\n \"description\": \"The SQL dialect that is used for the query. Should be compatible to the server.type.\",\n \"examples\": [\n \"athena\",\n \"bigquery\",\n \"redshift\",\n \"snowflake\",\n \"trino\",\n \"postgres\",\n \"oracle\"\n ]\n },\n \"mustBe\": {\n \"type\": \"integer\"\n },\n \"mustNotBe\": {\n \"type\": \"integer\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"integer\"\n },\n \"mustBeGreaterThanOrEqualTo\": {\n \"type\": \"integer\"\n },\n \"mustBeLessThan\": {\n \"type\": \"integer\"\n },\n \"mustBeLessThanOrEqualTo\": {\n \"type\": \"integer\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"integer\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"integer\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n }\n },\n \"required\": [\n \"query\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"library\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"rule\": {\n \"type\": \"string\",\n \"description\": \"Define a data quality check based on the predefined rules as per ODCS.\",\n \"examples\": [\"duplicateCount\", \"validValues\", \"rowCount\"]\n },\n \"mustBe\": {\n \"description\": \"Must be equal to the value to be valid. When using numbers, it is equivalent to '='.\"\n },\n \"mustNotBe\": {\n \"description\": \"Must not be equal to the value to be valid. When using numbers, it is equivalent to '!='.\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than the value to be valid. It is equivalent to '>'.\"\n },\n \"mustBeGreaterOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than or equal to the value to be valid. It is equivalent to '>='.\"\n },\n \"mustBeLessThan\": {\n \"type\": \"number\",\n \"description\": \"Must be less than the value to be valid. It is equivalent to '<'.\"\n },\n \"mustBeLessOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be less than or equal to the value to be valid. It is equivalent to '<='.\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must not be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n }\n },\n \"required\": [\n \"rule\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"custom\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n },\n \"engine\": {\n \"type\": \"string\",\n \"examples\": [\n \"soda\",\n \"great-expectations\"\n ],\n \"description\": \"The engine used for custom quality checks.\"\n },\n \"implementation\": {\n \"type\": [\n \"object\",\n \"array\",\n \"string\"\n ],\n \"description\": \"Engine-specific quality checks and expectations.\"\n }\n },\n \"required\": [\n \"engine\"\n ]\n }\n }\n ]\n },\n \"Lineage\": {\n \"type\": \"object\",\n \"properties\": {\n \"inputFields\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"namespace\": {\n \"type\": \"string\",\n \"description\": \"The input dataset namespace\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The input dataset name\"\n },\n \"field\": {\n \"type\": \"string\",\n \"description\": \"The input field\"\n },\n \"transformations\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"description\": \"The type of the transformation. Allowed values are: DIRECT, INDIRECT\",\n \"type\": \"string\"\n },\n \"subtype\": {\n \"type\": \"string\",\n \"description\": \"The subtype of the transformation\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\"\n },\n \"masking\": {\n \"type\": \"boolean\",\n \"description\": \"is transformation masking the data or not\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"additionalProperties\": true\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"namespace\",\n \"name\",\n \"field\"\n ]\n }\n },\n \"transformationDescription\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\",\n \"deprecated\": true\n },\n \"transformationType\": {\n \"type\": \"string\",\n \"description\": \"IDENTITY|MASKED reflects a clearly defined behavior. IDENTITY: exact same as input; MASKED: no original data available (like a hash of PII for example)\",\n \"deprecated\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"inputFields\"\n ]\n }\n }\n}\n" }, { "path": "versions/1.2.0/definition.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"properties\": {\n \"id\": {\n \"type\": \"string\",\n \"description\": \"A unique identifier for this definition. Encode the domain into the ID, separated by slashes.\",\n \"examples\": [\n \"checkout/order_id\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n}\n" }, { "path": "versions/1.2.1/datacontract.init.yaml", "content": "dataContractSpecification: 1.2.0\nid: my-data-contract-id\ninfo:\n title: My Data Contract\n version: 0.0.1\n# description:\n# owner:\n# contact:\n# name:\n# url:\n# email:\n\n\n### servers\n\n#servers:\n# production:\n# type: s3\n# location: s3://\n# format: parquet\n# delimiter: new_line\n\n### terms\n\n#terms:\n# usage:\n# limitations:\n# billing:\n# noticePeriod:\n\n\n### models\n\n# models:\n# my_model:\n# description:\n# type:\n# fields:\n# my_field:\n# type:\n# description:\n\n\n### definitions\n\n# definitions:\n# my_field:\n# domain:\n# name:\n# title:\n# type:\n# description:\n# example:\n# pii:\n# classification:\n\n\n### servicelevels\n\n#servicelevels:\n# availability:\n# description: The server is available during support hours\n# percentage: 99.9%\n# retention:\n# description: Data is retained for one year because!\n# period: P1Y\n# unlimited: false\n# latency:\n# description: Data is available within 25 hours after the order was placed\n# threshold: 25h\n# sourceTimestampField: orders.order_timestamp\n# processedTimestampField: orders.processed_timestamp\n# freshness:\n# description: The age of the youngest row in a table.\n# threshold: 25h\n# timestampField: orders.order_timestamp\n# frequency:\n# description: Data is delivered once a day\n# type: batch # or streaming\n# interval: daily # for batch, either or cron\n# cron: 0 0 * * * # for batch, either or interval\n# support:\n# description: The data is available during typical business hours at headquarters\n# time: 9am to 5pm in EST on business days\n# responseTime: 1h\n# backup:\n# description: Data is backed up once a week, every Sunday at 0:00 UTC.\n# interval: weekly\n# cron: 0 0 * * 0\n# recoveryTime: 24 hours\n# recoveryPoint: 1 week\n" }, { "path": "versions/1.2.1/datacontract.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"title\": \"DataContractSpecification\",\n \"properties\": {\n \"dataContractSpecification\": {\n \"type\": \"string\",\n \"title\": \"DataContractSpecificationVersion\",\n \"enum\": [\n \"1.2.1\",\n \"1.2.0\",\n \"1.1.0\",\n \"0.9.3\",\n \"0.9.2\",\n \"0.9.1\",\n \"0.9.0\"\n ],\n \"description\": \"Specifies the Data Contract Specification being used.\"\n },\n \"id\": {\n \"type\": \"string\",\n \"description\": \"Specifies the identifier of the data contract.\"\n },\n \"info\": {\n \"type\": \"object\",\n \"properties\": {\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The title of the data contract.\"\n },\n \"version\": {\n \"type\": \"string\",\n \"description\": \"The version of the data contract document (which is distinct from the Data Contract Specification version or the Data Product implementation version).\"\n },\n \"status\": {\n \"type\": \"string\",\n \"description\": \"The status of the data contract. Can be proposed, in development, active, retired.\",\n \"examples\": [\n \"proposed\",\n \"in development\",\n \"active\",\n \"deprecated\",\n \"retired\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the data contract.\"\n },\n \"owner\": {\n \"type\": \"string\",\n \"description\": \"The owner or team responsible for managing the data contract and providing the data.\"\n },\n \"contact\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The identifying name of the contact person/organization.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The URL pointing to the contact information. This MUST be in the form of a URL.\"\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"The email address of the contact person/organization. This MUST be in the form of an email address.\"\n }\n },\n \"description\": \"Contact information for the data contract.\",\n \"additionalProperties\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"title\",\n \"version\"\n ],\n \"description\": \"Metadata and life cycle information about the data contract.\"\n },\n \"servers\": {\n \"type\": \"object\",\n \"description\": \"Information about the servers.\",\n \"additionalProperties\": {\n \"$ref\": \"#/$defs/BaseServer\",\n \"allOf\": [\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"bigquery\"\n }\n }\n },\n \"then\": {\n \"$ref\": \"#/$defs/BigQueryServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"s3\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/S3Server\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sftp\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SftpServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"redshift\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/RedshiftServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"azure\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/AzureServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sqlserver\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SqlserverServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"snowflake\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/SnowflakeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"databricks\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DatabricksServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"dataframe\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/DataframeServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"glue\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/GlueServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"postgres\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PostgresServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"oracle\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/OracleServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kafka\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KafkaServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"pubsub\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/PubSubServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"kinesis\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/KinesisDataStreamsServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"trino\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/TrinoServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"clickhouse\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/ClickhouseServer\"\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"local\"\n }\n },\n \"required\": [\n \"type\"\n ]\n },\n \"then\": {\n \"$ref\": \"#/$defs/LocalServer\"\n }\n }\n ]\n }\n },\n \"terms\": {\n \"type\": \"object\",\n \"description\": \"The terms and conditions of the data contract.\",\n \"properties\": {\n \"usage\": {\n \"type\": \"string\",\n \"description\": \"The usage describes the way the data is expected to be used. Can contain business and technical information.\"\n },\n \"limitations\": {\n \"type\": \"string\",\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"policies\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the policy.\",\n \"examples\": [\n \"privacy\",\n \"security\",\n \"retention\",\n \"compliance\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the policy.\"\n },\n \"url\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"A URL to the policy document.\"\n }\n },\n \"additionalProperties\": true\n },\n \"description\": \"The limitations describe the restrictions on how the data can be used, can be technical or restrictions on what the data may not be used for.\"\n },\n \"billing\": {\n \"type\": \"string\",\n \"description\": \"The billing describes the pricing model for using the data, such as whether it's free, having a monthly fee, or metered pay-per-use.\"\n },\n \"noticePeriod\": {\n \"type\": \"string\",\n \"description\": \"The period of time that must be given by either party to terminate or modify a data usage agreement. Uses ISO-8601 period format, e.g., 'P3M' for a period of three months.\"\n }\n },\n \"additionalProperties\": true\n },\n \"models\": {\n \"description\": \"Specifies the logical data model. Use the models name (e.g., the table name) as the key.\",\n \"type\": \"object\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Model\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"The type of the model. Examples: table, view, object. Default: table.\",\n \"type\": \"string\",\n \"title\": \"ModelType\",\n \"default\": \"table\",\n \"enum\": [\n \"table\",\n \"view\",\n \"object\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the model. Especially useful if the model name is cryptic or contains abbreviations.\",\n \"examples\": [\n \"Purchase Orders\",\n \"Air Shipments\"\n ]\n },\n \"fields\": {\n \"description\": \"Specifies a field in the data model. Use the field name (e.g., the column name) as the key.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Field\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the semantic of the data in this field.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"An optional string providing a human readable name for the field. Especially useful if the field name is cryptic or contains abbreviations.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"required\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if this field must contain a value and may not be null.\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"primary\": {\n \"type\": \"boolean\",\n \"deprecationMessage\": \"Use the primaryKey field instead.\"\n },\n \"primaryKey\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"If this field is a primary key.\"\n },\n \"references\": {\n \"type\": \"string\",\n \"description\": \"The reference to a field in another model. E.g. use 'orders.order_id' to reference the order_id field of the model orders. Think of defining a foreign key relationship.\",\n \"examples\": [\n \"orders.order_id\",\n \"model.nested_field.field\"\n ]\n },\n \"unique\": {\n \"type\": \"boolean\",\n \"default\": false,\n \"description\": \"An indication, if the value must be unique within the model.\"\n },\n \"enum\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"uniqueItems\": true,\n \"description\": \"A value must be equal to one of the elements in this array value. Only evaluated if the value is not null.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must greater than, or equal to, the value of this. Only applies to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must less than, or equal to, the value of this. Only applies to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"A specific format the value must comply with (e.g., 'email', 'uri', 'uuid').\",\n \"examples\": [\n \"email\",\n \"uri\",\n \"uuid\"\n ]\n },\n \"precision\": {\n \"type\": \"number\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"number\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression the value must match. Only applies to string types.\",\n \"examples\": [\n \"^[a-zA-Z0-9_-]+$\"\n ]\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"An indication, if this field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field, according to the organization's classification scheme.\",\n \"examples\": [\n \"sensitive\",\n \"restricted\",\n \"internal\",\n \"public\"\n ]\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"$ref\": {\n \"type\": \"string\",\n \"description\": \"A reference URI to a definition in the specification, internally or externally. Properties will be inherited from the definition.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"lineage\": {\n \"$ref\": \"#/$defs/Lineage\"\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for field configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroType\": {\n \"type\": \"string\",\n \"description\": \"Specify the field type to use when exporting the data model to Apache Avro.\"\n },\n \"avroLogicalType\": {\n \"type\": \"string\",\n \"description\": \"Specify the logical field type to use when exporting the data model to Apache Avro.\"\n },\n \"bigqueryType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a BigQuery table, e.g., `NUMERIC(5, 2)`.\"\n },\n \"snowflakeType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Snowflake table, e.g., `TIMESTAMP_LTZ`.\"\n },\n \"redshiftType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Redshift table, e.g., `SMALLINT`.\"\n },\n \"sqlserverType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a SQL Server table, e.g., `DATETIME2`.\"\n },\n \"databricksType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in a Databricks Unity Catalog table.\"\n },\n \"glueType\": {\n \"type\": \"string\",\n \"description\": \"Specify the physical column type that is used in an AWS Glue Data Catalog table.\"\n }\n }\n }\n }\n }\n },\n \"primaryKey\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"The compound primary key of the model.\"\n },\n \"quality\": {\n \"type\": \"array\",\n \"items\": {\n \"$ref\": \"#/$defs/Quality\"\n }\n },\n \"examples\": {\n \"type\": \"array\"\n },\n \"additionalFields\": {\n \"type\": \"boolean\",\n \"description\": \" Specify, if the model can have additional fields that are not defined in the contract. \",\n \"default\": false\n },\n \"config\": {\n \"type\": \"object\",\n \"description\": \"Additional metadata for model configuration.\",\n \"additionalProperties\": {\n \"type\": [\n \"string\",\n \"number\",\n \"boolean\",\n \"object\",\n \"array\",\n \"null\"\n ]\n },\n \"properties\": {\n \"avroNamespace\": {\n \"type\": \"string\",\n \"description\": \"The namespace to use when importing and exporting the data model from / to Apache Avro.\"\n }\n }\n }\n }\n }\n },\n \"definitions\": {\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"type\": \"object\",\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9/_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"object\",\n \"title\": \"Definition\",\n \"properties\": {\n \"domain\": {\n \"type\": \"string\",\n \"description\": \"The domain in which this definition is valid.\",\n \"default\": \"global\",\n \"deprecationMessage\": \"This field is deprecated. Encode the domain into the ID using slashes.\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The technical name of this definition.\",\n \"deprecationMessage\": \"This field is deprecated. Encode the name into the ID using slashes.\"\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"$ref\": \"#/$defs/FieldType\"\n },\n \"fields\": {\n \"description\": \"The nested fields (e.g. columns) of the object, record, or struct.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n }\n },\n \"items\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"keys\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"values\": {\n \"$ref\": \"#/properties/models/additionalProperties/properties/fields/additionalProperties\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"minimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMinimum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must greater than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"maximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than, or equal to, the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"exclusiveMaximum\": {\n \"type\": \"number\",\n \"description\": \"A value of a number must less than the value of this. Only evaluated if the value is not null. Only applies to numeric values.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"Example value.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n }\n },\n \"servicelevels\": {\n \"type\": \"object\",\n \"description\": \"Specifies the service level agreements for the provided data, including availability, data retention policies, latency requirements, data freshness, update frequency, support availability, and backup policies.\",\n \"properties\": {\n \"availability\": {\n \"type\": \"object\",\n \"description\": \"Availability refers to the promise or guarantee by the service provider about the uptime of the system that provides the data.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the availability service level.\",\n \"example\": \"The server is available during support hours\"\n },\n \"percentage\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the guaranteed uptime in percent (e.g., `99.9%`)\",\n \"pattern\": \"^\\\\d+(\\\\.\\\\d+)?%$\",\n \"example\": \"99.9%\"\n }\n }\n },\n \"retention\": {\n \"type\": \"object\",\n \"description\": \"Retention covers the period how long data will be available.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the retention service level.\",\n \"example\": \"Data is retained for one year.\"\n },\n \"period\": {\n \"type\": \"string\",\n \"description\": \"An optional period of time, how long data is available. Supported formats: Simple duration (e.g., `1 year`, `30d`) and ISO 8601 duration (e.g, `P1Y`).\",\n \"example\": \"P1Y\"\n },\n \"unlimited\": {\n \"type\": \"boolean\",\n \"description\": \"An optional indicator that data is kept forever.\",\n \"example\": false\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the period refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"latency\": {\n \"type\": \"object\",\n \"description\": \"Latency refers to the maximum amount of time from the source to its destination.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the latency service level.\",\n \"example\": \"Data is available within 25 hours after the order was placed.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum duration between the source timestamp and the processed timestamp. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g, `PT24H`).\",\n \"example\": \"25h\"\n },\n \"sourceTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp when the data was provided at the source.\",\n \"example\": \"orders.order_timestamp\"\n },\n \"processedTimestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the processing timestamp, which denotes when the data is made available to consumers of this data contract.\",\n \"example\": \"orders.processed_timestamp\"\n }\n }\n },\n \"freshness\": {\n \"type\": \"object\",\n \"description\": \"The maximum age of the youngest row in a table.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the freshness service level.\",\n \"example\": \"The age of the youngest row in a table is within 25 hours.\"\n },\n \"threshold\": {\n \"type\": \"string\",\n \"description\": \"An optional maximum age of the youngest entry. Supported formats: Simple duration (e.g., `24 hours`, `5s`) and ISO 8601 duration (e.g., `PT24H`).\",\n \"example\": \"25h\"\n },\n \"timestampField\": {\n \"type\": \"string\",\n \"description\": \"An optional reference to the field that contains the timestamp that the threshold refers to.\",\n \"example\": \"orders.order_timestamp\"\n }\n }\n },\n \"frequency\": {\n \"type\": \"object\",\n \"description\": \"Frequency describes how often data is updated.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the frequency service level.\",\n \"example\": \"Data is delivered once a day.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"enum\": [\n \"batch\",\n \"micro-batching\",\n \"streaming\",\n \"manual\"\n ],\n \"description\": \"The method of data processing.\",\n \"example\": \"batch\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: How often the pipeline is triggered, e.g., `daily`.\",\n \"example\": \"daily\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"Optional. Only for batch: A cron expression when the pipelines is triggered. E.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * *\"\n }\n }\n },\n \"support\": {\n \"type\": \"object\",\n \"description\": \"Support describes the times when support will be available for contact.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the support service level.\",\n \"example\": \"The data is available during typical business hours at headquarters.\"\n },\n \"time\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the times when support will be available for contact such as `24/7` or `business hours only`.\",\n \"example\": \"9am to 5pm in EST on business days\"\n },\n \"responseTime\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the time it takes for the support team to acknowledge a request. This does not mean the issue will be resolved immediately, but it assures users that their request has been received and will be dealt with.\",\n \"example\": \"24 hours\"\n }\n }\n },\n \"backup\": {\n \"type\": \"object\",\n \"description\": \"Backup specifies details about data backup procedures.\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the backup service level.\",\n \"example\": \"Data is backed up once a week, every Sunday at 0:00 UTC.\"\n },\n \"interval\": {\n \"type\": \"string\",\n \"description\": \"An optional interval that defines how often data will be backed up, e.g., `daily`.\",\n \"example\": \"weekly\"\n },\n \"cron\": {\n \"type\": \"string\",\n \"description\": \"An optional cron expression when data will be backed up, e.g., `0 0 * * *`.\",\n \"example\": \"0 0 * * 0\"\n },\n \"recoveryTime\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Time Objective (RTO) specifies the maximum amount of time allowed to restore data from a backup after a failure or loss event (e.g., 4 hours, 24 hours).\",\n \"example\": \"24 hours\"\n },\n \"recoveryPoint\": {\n \"type\": \"string\",\n \"description\": \"An optional Recovery Point Objective (RPO) defines the maximum acceptable age of files that must be recovered from backup storage for normal operations to resume after a disaster or data loss event. This essentially measures how much data you can afford to lose, measured in time (e.g., 4 hours, 24 hours).\",\n \"example\": \"1 week\"\n }\n }\n }\n }\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\",\n \"description\": \"Tags to facilitate searching and filtering.\",\n \"examples\": [\n \"databricks\",\n \"pii\",\n \"sensitive\"\n ]\n },\n \"description\": \"Tags to facilitate searching and filtering.\"\n }\n },\n \"required\": [\n \"dataContractSpecification\",\n \"id\",\n \"info\"\n ],\n \"$defs\": {\n \"FieldType\": {\n \"type\": \"string\",\n \"title\": \"FieldType\",\n \"description\": \"The logical data type of the field.\",\n \"enum\": [\n \"number\",\n \"decimal\",\n \"numeric\",\n \"int\",\n \"integer\",\n \"long\",\n \"bigint\",\n \"float\",\n \"double\",\n \"string\",\n \"text\",\n \"varchar\",\n \"boolean\",\n \"timestamp\",\n \"timestamp_tz\",\n \"timestamp_ntz\",\n \"date\",\n \"time\",\n \"array\",\n \"map\",\n \"object\",\n \"record\",\n \"struct\",\n \"bytes\",\n \"variant\",\n \"json\",\n \"null\"\n ]\n },\n \"BaseServer\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the servers.\"\n },\n \"environment\": {\n \"type\": \"string\",\n \"description\": \"The environment in which the servers are running. Examples: prod, sit, stg.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of the data product technology that implements the data contract.\",\n \"examples\": [\n \"azure\",\n \"bigquery\",\n \"BigQuery\",\n \"clickhouse\",\n \"databricks\",\n \"dataframe\",\n \"glue\",\n \"kafka\",\n \"kinesis\",\n \"local\",\n \"oracle\",\n \"postgres\",\n \"pubsub\",\n \"redshift\",\n \"sftp\",\n \"sqlserver\",\n \"snowflake\",\n \"s3\",\n \"trino\"\n ]\n },\n \"roles\": {\n \"description\": \" An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The name of the role.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A description of the role and what access the role provides.\"\n }\n },\n \"required\": [\n \"name\"\n ]\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"type\"\n ]\n },\n \"BigQueryServer\": {\n \"type\": \"object\",\n \"title\": \"BigQueryServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"dataset\": {\n \"type\": \"string\",\n \"description\": \"The GCP dataset name.\"\n }\n },\n \"required\": [\n \"project\",\n \"dataset\"\n ]\n },\n \"S3Server\": {\n \"type\": \"object\",\n \"title\": \"S3Server\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"S3 URL, starting with `s3://`\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}/*.json\"\n ]\n },\n \"endpointUrl\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The server endpoint for S3-compatible servers.\",\n \"examples\": [\n \"https://minio.example.com\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"SftpServer\": {\n \"type\": \"object\",\n \"title\": \"SftpServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"pattern\": \"^sftp://.*\",\n \"description\": \"SFTP URL, starting with `sftp://`\",\n \"examples\": [\n \"sftp://123.123.12.123/{model}/*.json\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\"\n ]\n },\n \"RedshiftServer\": {\n \"type\": \"object\",\n \"title\": \"RedshiftServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"host\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the host name.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"clusterIdentifier\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's identifier.\",\n \"examples\": [\n \"redshift-prod-eu\",\n \"analytics-cluster\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"An optional string describing the cluster's port.\",\n \"examples\": [\n 5439\n ]\n },\n \"endpoint\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the cluster's endpoint.\",\n \"examples\": [\n \"analytics-cluster.example.eu-west-1.redshift.amazonaws.com:5439/analytics\"\n ]\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"AzureServer\": {\n \"type\": \"object\",\n \"title\": \"AzureServer\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"Path to Azure Blob Storage or Azure Data Lake Storage (ADLS), supports globs. Recommended pattern is 'abfss:///'\",\n \"examples\": [\n \"abfss://my_container_name/path\",\n \"abfss://my_container_name/path/*.json\",\n \"az://my_storage_account_name.blob.core.windows.net/my_container/path/*.parquet\",\n \"abfss://my_storage_account_name.dfs.core.windows.net/my_container_name/path/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"enum\": [\n \"parquet\",\n \"delta\",\n \"json\",\n \"csv\"\n ],\n \"description\": \"File format.\"\n },\n \"delimiter\": {\n \"type\": \"string\",\n \"enum\": [\n \"new_line\",\n \"array\"\n ],\n \"description\": \"Only for format = json. How multiple json documents are delimited within one file\"\n }\n },\n \"required\": [\n \"location\",\n \"format\"\n ]\n },\n \"SqlserverServer\": {\n \"type\": \"object\",\n \"title\": \"SqlserverServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\",\n \"default\": 1433,\n \"examples\": [\n 1433\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"database\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"dbo\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"database\",\n \"schema\"\n ]\n },\n \"SnowflakeServer\": {\n \"type\": \"object\",\n \"title\": \"SnowflakeServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"An optional string describing the server.\"\n }\n },\n \"required\": [\n \"account\",\n \"database\",\n \"schema\"\n ]\n },\n \"DatabricksServer\": {\n \"type\": \"object\",\n \"title\": \"DatabricksServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Databricks host\",\n \"examples\": [\n \"dbc-abcdefgh-1234.cloud.databricks.com\"\n ]\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the Hive or Unity catalog\"\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The schema name in the catalog\"\n }\n },\n \"required\": [\n \"catalog\",\n \"schema\"\n ]\n },\n \"DataframeServer\": {\n \"type\": \"object\",\n \"title\": \"DataframeServer\",\n \"required\": [\n \"type\"\n ]\n },\n \"GlueServer\": {\n \"type\": \"object\",\n \"title\": \"GlueServer\",\n \"properties\": {\n \"account\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue account\",\n \"examples\": [\n \"1234-5678-9012\"\n ]\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The AWS Glue database name\",\n \"examples\": [\n \"my_database\"\n ]\n },\n \"location\": {\n \"type\": \"string\",\n \"format\": \"uri\",\n \"description\": \"The AWS S3 path. Must be in the form of a URL.\",\n \"examples\": [\n \"s3://datacontract-example-orders-latest/data/{model}\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the files\",\n \"examples\": [\n \"parquet\",\n \"csv\",\n \"json\",\n \"delta\"\n ]\n }\n },\n \"required\": [\n \"account\",\n \"database\"\n ]\n },\n \"PostgresServer\": {\n \"type\": \"object\",\n \"title\": \"PostgresServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"public\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\",\n \"schema\"\n ]\n },\n \"OracleServer\": {\n \"type\": \"object\",\n \"title\": \"OracleServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the oracle server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the oracle server.\",\n \"examples\": [\n 1523\n ]\n },\n \"serviceName\": {\n \"type\": \"string\",\n \"description\": \"The name of the service.\",\n \"examples\": [\n \"service\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"serviceName\"\n ]\n },\n \"KafkaServer\": {\n \"type\": \"object\",\n \"title\": \"KafkaServer\",\n \"description\": \"Kafka Server\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The bootstrap server of the kafka cluster.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the message. Examples: json, avro, protobuf.\",\n \"default\": \"json\"\n }\n },\n \"required\": [\n \"host\",\n \"topic\"\n ]\n },\n \"PubSubServer\": {\n \"type\": \"object\",\n \"title\": \"PubSubServer\",\n \"properties\": {\n \"project\": {\n \"type\": \"string\",\n \"description\": \"The GCP project name.\"\n },\n \"topic\": {\n \"type\": \"string\",\n \"description\": \"The topic name.\"\n }\n },\n \"required\": [\n \"project\",\n \"topic\"\n ]\n },\n \"KinesisDataStreamsServer\": {\n \"type\": \"object\",\n \"title\": \"KinesisDataStreamsServer\",\n \"description\": \"Kinesis Data Streams Server\",\n \"properties\": {\n \"stream\": {\n \"type\": \"string\",\n \"description\": \"The name of the Kinesis data stream.\"\n },\n \"region\": {\n \"type\": \"string\",\n \"description\": \"AWS region.\",\n \"examples\": [\n \"eu-west-1\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the record\",\n \"examples\": [\n \"json\",\n \"avro\",\n \"protobuf\"\n ]\n }\n },\n \"required\": [\n \"stream\"\n ]\n },\n \"TrinoServer\": {\n \"type\": \"object\",\n \"title\": \"TrinoServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The Trino host URL.\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The Trino port.\"\n },\n \"catalog\": {\n \"type\": \"string\",\n \"description\": \"The name of the catalog.\",\n \"examples\": [\n \"hive\"\n ]\n },\n \"schema\": {\n \"type\": \"string\",\n \"description\": \"The name of the schema in the database.\",\n \"examples\": [\n \"my_schema\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"catalog\",\n \"schema\"\n ]\n },\n \"ClickhouseServer\": {\n \"type\": \"object\",\n \"title\": \"ClickhouseServer\",\n \"properties\": {\n \"host\": {\n \"type\": \"string\",\n \"description\": \"The host to the database server\",\n \"examples\": [\n \"localhost\"\n ]\n },\n \"port\": {\n \"type\": \"integer\",\n \"description\": \"The port to the database server.\"\n },\n \"database\": {\n \"type\": \"string\",\n \"description\": \"The name of the database.\",\n \"examples\": [\n \"postgres\"\n ]\n }\n },\n \"required\": [\n \"host\",\n \"port\",\n \"database\"\n ]\n },\n \"LocalServer\": {\n \"type\": \"object\",\n \"title\": \"LocalServer\",\n \"properties\": {\n \"path\": {\n \"type\": \"string\",\n \"description\": \"The relative or absolute path to the data file(s).\",\n \"examples\": [\n \"./folder/data.parquet\",\n \"./folder/*.parquet\"\n ]\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"The format of the file(s)\",\n \"examples\": [\n \"json\",\n \"parquet\",\n \"delta\",\n \"csv\"\n ]\n }\n },\n \"required\": [\n \"path\",\n \"format\"\n ]\n },\n \"Quality\": {\n \"allOf\": [\n {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The type of quality check\",\n \"enum\": [\n \"text\",\n \"library\",\n \"sql\",\n \"custom\"\n ]\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n }\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"text\"\n }\n }\n },\n \"then\": {\n \"required\": [\n \"description\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"sql\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"query\": {\n \"type\": \"string\",\n \"description\": \"A SQL query that returns a single number to compare with the threshold.\"\n },\n \"dialect\": {\n \"type\": \"string\",\n \"description\": \"The SQL dialect that is used for the query. Should be compatible to the server.type.\",\n \"examples\": [\n \"athena\",\n \"bigquery\",\n \"redshift\",\n \"snowflake\",\n \"trino\",\n \"postgres\",\n \"oracle\"\n ]\n },\n \"mustBe\": {\n \"type\": \"number\"\n },\n \"mustNotBe\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterOrEqualTo\": {\n \"type\": \"number\"\n },\n \"mustBeGreaterThanOrEqualTo\": {\n \"type\": \"number\",\n \"deprecated\": true\n },\n \"mustBeLessThan\": {\n \"type\": \"number\"\n },\n \"mustBeLessThanOrEqualTo\": {\n \"type\": \"number\",\n \"deprecated\": true\n },\n \"mustBeLessOrEqualTo\": {\n \"type\": \"number\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"number\"\n },\n \"minItems\": 2,\n \"maxItems\": 2\n }\n },\n \"required\": [\n \"query\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"library\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"metric\": {\n \"type\": \"string\",\n \"description\": \"The DataQualityLibrary metric to use for the quality check.\",\n \"examples\": [\"nullValues\", \"missingValues\", \"invalidValues\", \"duplicateValues\", \"rowCount\"]\n },\n \"rule\": {\n \"type\": \"string\",\n \"deprecated\": true,\n \"description\": \"Deprecated. Use metric instead\"\n },\n \"arguments\": {\n \"type\": \"object\",\n \"description\": \"Additional metric-specific parameters for the quality check.\",\n \"additionalProperties\": {\n \"type\": [\"string\", \"number\", \"boolean\", \"array\", \"object\"]\n }\n },\n \"mustBe\": {\n \"description\": \"Must be equal to the value to be valid. When using numbers, it is equivalent to '='.\"\n },\n \"mustNotBe\": {\n \"description\": \"Must not be equal to the value to be valid. When using numbers, it is equivalent to '!='.\"\n },\n \"mustBeGreaterThan\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than the value to be valid. It is equivalent to '>'.\"\n },\n \"mustBeGreaterOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be greater than or equal to the value to be valid. It is equivalent to '>='.\"\n },\n \"mustBeLessThan\": {\n \"type\": \"number\",\n \"description\": \"Must be less than the value to be valid. It is equivalent to '<'.\"\n },\n \"mustBeLessOrEqualTo\": {\n \"type\": \"number\",\n \"description\": \"Must be less than or equal to the value to be valid. It is equivalent to '<='.\"\n },\n \"mustBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n },\n \"mustNotBeBetween\": {\n \"type\": \"array\",\n \"description\": \"Must not be between the two numbers to be valid. Smallest number first in the array.\",\n \"minItems\": 2,\n \"maxItems\": 2,\n \"uniqueItems\": true,\n \"items\": {\n \"type\": \"number\"\n }\n }\n },\n \"required\": [\n \"metric\"\n ]\n }\n },\n {\n \"if\": {\n \"properties\": {\n \"type\": {\n \"const\": \"custom\"\n }\n }\n },\n \"then\": {\n \"properties\": {\n \"description\": {\n \"type\": \"string\",\n \"description\": \"A plain text describing the quality attribute in natural language.\"\n },\n \"engine\": {\n \"type\": \"string\",\n \"examples\": [\n \"soda\",\n \"great-expectations\"\n ],\n \"description\": \"The engine used for custom quality checks.\"\n },\n \"implementation\": {\n \"type\": [\n \"object\",\n \"array\",\n \"string\"\n ],\n \"description\": \"Engine-specific quality checks and expectations.\"\n }\n },\n \"required\": [\n \"engine\"\n ]\n }\n }\n ]\n },\n \"Lineage\": {\n \"type\": \"object\",\n \"properties\": {\n \"inputFields\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"namespace\": {\n \"type\": \"string\",\n \"description\": \"The input dataset namespace\"\n },\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The input dataset name\"\n },\n \"field\": {\n \"type\": \"string\",\n \"description\": \"The input field\"\n },\n \"transformations\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"type\": {\n \"description\": \"The type of the transformation. Allowed values are: DIRECT, INDIRECT\",\n \"type\": \"string\"\n },\n \"subtype\": {\n \"type\": \"string\",\n \"description\": \"The subtype of the transformation\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\"\n },\n \"masking\": {\n \"type\": \"boolean\",\n \"description\": \"is transformation masking the data or not\"\n }\n },\n \"required\": [\n \"type\"\n ],\n \"additionalProperties\": true\n }\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"namespace\",\n \"name\",\n \"field\"\n ]\n }\n },\n \"transformationDescription\": {\n \"type\": \"string\",\n \"description\": \"a string representation of the transformation applied\",\n \"deprecated\": true\n },\n \"transformationType\": {\n \"type\": \"string\",\n \"description\": \"IDENTITY|MASKED reflects a clearly defined behavior. IDENTITY: exact same as input; MASKED: no original data available (like a hash of PII for example)\",\n \"deprecated\": true\n }\n },\n \"additionalProperties\": true,\n \"required\": [\n \"inputFields\"\n ]\n }\n }\n}\n" }, { "path": "versions/1.2.1/definition.schema.json", "content": "{\n \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n \"type\": \"object\",\n \"description\": \"Clear and concise explanations of syntax, semantic, and classification of business objects in a given domain.\",\n \"properties\": {\n \"id\": {\n \"type\": \"string\",\n \"description\": \"A unique identifier for this definition. Encode the domain into the ID, separated by slashes.\",\n \"examples\": [\n \"checkout/order_id\"\n ]\n },\n \"title\": {\n \"type\": \"string\",\n \"description\": \"The business name of this definition.\"\n },\n \"description\": {\n \"type\": \"string\",\n \"description\": \"Clear and concise explanations related to the domain.\"\n },\n \"type\": {\n \"type\": \"string\",\n \"description\": \"The logical data type.\"\n },\n \"minLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be greater than or equal to this value. Applies only to string types.\"\n },\n \"maxLength\": {\n \"type\": \"integer\",\n \"description\": \"A value must be less than or equal to this value. Applies only to string types.\"\n },\n \"format\": {\n \"type\": \"string\",\n \"description\": \"Specific format requirements for the value (e.g., 'email', 'uri', 'uuid').\"\n },\n \"precision\": {\n \"type\": \"integer\",\n \"examples\": [\n 38\n ],\n \"description\": \"The maximum number of digits in a number. Only applies to numeric values. Defaults to 38.\"\n },\n \"scale\": {\n \"type\": \"integer\",\n \"examples\": [\n 0\n ],\n \"description\": \"The maximum number of decimal places in a number. Only applies to numeric values. Defaults to 0.\"\n },\n \"pattern\": {\n \"type\": \"string\",\n \"description\": \"A regular expression pattern the value must match. Applies only to string types.\"\n },\n \"example\": {\n \"type\": \"string\",\n \"description\": \"An example value for this field.\",\n \"deprecationMessage\": \"Use the examples field instead.\"\n },\n \"examples\": {\n \"type\": \"array\",\n \"description\": \"A examples value for this field.\"\n },\n \"pii\": {\n \"type\": \"boolean\",\n \"description\": \"Indicates if the field contains Personal Identifiable Information (PII).\"\n },\n \"classification\": {\n \"type\": \"string\",\n \"description\": \"The data class defining the sensitivity level for this field.\"\n },\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n },\n \"description\": \"Custom metadata to provide additional context.\"\n },\n \"links\": {\n \"type\": \"object\",\n \"description\": \"Links to external resources.\",\n \"minProperties\": 1,\n \"propertyNames\": {\n \"pattern\": \"^[a-zA-Z0-9_-]+$\"\n },\n \"additionalProperties\": {\n \"type\": \"string\",\n \"title\": \"Link\",\n \"description\": \"A URL to an external resource.\",\n \"format\": \"uri\",\n \"examples\": [\n \"https://example.com\"\n ]\n }\n }\n },\n \"required\": [\n \"type\"\n ]\n}\n" }, { "path": "workshop.md", "content": "# Data Contract Workshop\n\nBring data producers and consumers together to define data contracts in a facilitated workshop.\n\n## Goal\n\nA defined and agreed upon data contract between data producers and consumers.\n\n## Participants\n\n- Facilitator\n - Neutral moderator and typist\n - Should know the used data contract formal ([Data Contract Specification](https://datacontract.com) or [ODCS](https://bitol-io.github.io/open-data-contract-standard/latest/)) and its tools well\n - Get the [authors of the Data Contract Specification](https://datacontract.com/#authors) as facilitators for your workshop.\n- Data producer\n - Product Owner\n - Software Engineers\n- Data consumers\n - Product Owner\n - Data Engineers / Scientist / Analyst\n\nRecommendation: keep the group small (not more than 5 people)\n\n## Settings\n\n- Show data contract the whole workshop on the screen (projector, screenshare, ...)\n- Facilitator is the typist\n- Facilitator is moderator\n- Data Producer and Data Consumers discuss and give commands to the facilitator\n\n## Guidelines for the Data Contract Specification\n\n### Recommended Order of Completion (Data Contract Specification)\n\n1. Info (get the context)\n2. Examples (example-driven facilitation)\n3. Model (you will spend most of your time here)\n - Use the [Data Contract CLI](https://cli.datacontract.com) to test the model against the previously created examples:\\\\\n `datacontract test --examples datacontract.yaml`\n4. Quality\n5. Terms\n6. Servers (if already applicable)\n - Start with a \"local\" server with actual, real data you downloaded\n - Use the [Data Contract CLI](https://cli.datacontract.com) to test the model against the actual data on a specific server:\\\\\n `datacontract test datacontract.yaml`\n - Switch to the actual remote server, if applicable\n\n### Tooling (Data Contract Specification)\n\n- Open the [starter template](https://datacontract.com/datacontract.init.yaml) in the [Data Contract Editor](https://editor.datacontract.com) and get going. If you lack an experienced facilitator, ignore any validation errors and warnings within the editor.\n- Use the [Data Contract Editor](https://editor.datacontract.com) to share the results of the workshop afterward with the participants and other stakeholders.\n- Use the [Data Contract CLI](https://cli.datacontract.com) to validate the data contract after the workshop.\n- Use the [Data Mesh Manager](https://www.datamesh-manager.com) to publish the data contract and have it in a central place\n\n## Guidelines for ODCS\n\nWe recommend to use the [Excel template](https://github.com/datacontract/open-data-contract-standard-excel-template) for workshops as it is easier to work with in such a setting as it comes with a nice visualization.\n\n### Recommended Order of Completion (ODCS)\n\n1. Fundamentals (get the context)\n - **[Fill in the fundamentals](https://bitol-io.github.io/open-data-contract-standard/latest/#fundamentals)** consisting of id, name, version, status, and description.\n2. Schema (you will spend most of your time here)\n - **[Fill in the schemas](https://bitol-io.github.io/open-data-contract-standard/latest/#schema)** (tables) and their properties (columns) along with their name and logicalType as a start in the schema part. \n - After that, add information like `description`, `classification`, ...\n - Use tags or customProperties add additional metadata where there is no direct support by ODCS\n3. Quality\n - **[Add quality checks](https://bitol-io.github.io/open-data-contract-standard/latest/#data-quality)** at the schema or the property level. Start with quality checks of type text first to capture the requirements.\n - OPTIONAL Conver the text-based requirements into automated sql-based quality checks\n4. SLAs\n - **[Add SLAs](https://bitol-io.github.io/open-data-contract-standard/latest/#service-level-agreement-sla)** that the data provider guarantees towards all data consumers.\n5. Team & Support\n - **[Add the team members](https://bitol-io.github.io/open-data-contract-standard/latest/#team)** so that the data consumer knows who is part of the team that owns the data protected by the data contracts.\n - **[Add a support channel](https://bitol-io.github.io/open-data-contract-standard/latest/#support-and-communication-channels)** so (potential) data consumers know how to get support and reach the data owners.\n6. Servers (if already applicable)\n - **[Add the server information](https://bitol-io.github.io/open-data-contract-standard/latest/#infrastructure-and-servers)** on where the data is available\n - Use the [Data Contract CLI](https://cli.datacontract.com) to test the schema against the actual data on a specific server:\\\\\n `datacontract test datacontract.yaml`\n\n### Tooling (ODCS)\n\n- Use the [Excel template](https://github.com/datacontract/open-data-contract-standard-excel-template) for the workshop\n- Use the [Data Contract CLI](https://cli.datacontract.com) to validate the data contract after the workshop.\n- Use the [Data Mesh Manager](https://www.datamesh-manager.com) to publish the data contract and have it in a central place\n\n## Related\n\n- This data contract workshop could be a followup to a data product design workshop using the [Data Product Canvas](https://www.datamesh-architecture.com/data-product-canvas), making the offered contract at the output port of the designed data product more concrete.\n" } ]