Changelog#

Unreleased#

Added#

New CLI options:

mapping-sheet: --no-inherit-extension

Changed#

Drop support for Python 3.6 (end-of-life 2021-12-23).

Fixed#

ocdskit.mapping_sheet.mapping_sheet() works if a schema contains "deprecated": null.
ocdskit.mapping_sheet.mapping_sheet() works if --extension-field is set to a value other than “extension”.

1.1.3 (2022-10-20)#

Fixed#

ocdskit.mapping_sheet.mapping_sheet() works if include_definitions=False but base_uri is not provided.

1.1.2 (2022-10-06)#

Changed#

Move ocdskit/cli/__main__.py to ocdskit/__main__.py, to support the python -m ocdskit interface.
Move ocdskit.cli.commands to ocdskit.commands.

1.1.1 (2022-08-17)#

Changed#

ocdskit.mapping_sheet.mapping_sheet() resolves $ref properties if include_definitions=False.

1.1.0 (2022-08-16)#

Changed#

ocdskit.mapping_sheet.mapping_sheet() returns columns and rows instead of writing to a file-like object.

1.0.4 (2022-02-10)#

Added#

New CLI options:

mapping-sheet: --codelist

1.0.3 (2021-12-18)#

Added#

ocdskit.util.get_ocds_patch_tag()

Changed#

compile omits packages from a record package for OCDS 1.2+.

Fixed#

compile omits packages from a record package if empty.
compile raises an error if the OCDS version is not recognized, instead of failing silently.

1.0.2 (2021-06-29)#

Fixed#

mapping-sheet correctly populates the extension column for extension fields on OCDS objects with items properties that $ref’erence OCDS definitions.

1.0.1 (2021-06-16)#

Changed#

ocdskit.util.detect_format() accepts a reader keyword argument: for example, gzip.open instead of open.

1.0.0 (2021-05-19)#

Changed#

validate: Remove command. Instead, use lib-cove-ocds.
tabulate: Remove command. Instead, convert the JSON data to CSV format using Spoonbill or Flatten Tool, and then load the CSV files into your preferred database. See:
- SQLite’s .import command (see also sqlite-utils)
- PostgreSQL’s COPY command
- MySQL’s LOAD DATA command
- csvkit’s csvsql command
convert-to-oc4ids: Remove command. Instead, use oc4idskit’s convert-from-ocds.
ocdskit.oc4ids: Remove module. Instead, use oc4idskit’s transforms.

0.2.23 (2021-05-06)#

Fixed#

mapping-sheet: Set deprecated on the extra rows for arrays.

0.2.22 (2021-04-23)#

Fixed#

schema-strict: Don’t add "uniqueItems": true to coordinates fields.

0.2.21 (2021-04-10)#

Added#

Add Python wheels distribution.

0.2.20 (2021-02-19)#

Added#

New CLI options:

mapping-sheet: --language

0.2.19 (2021-02-16)#

Fixed#

upgrade: If a party’s roles field isn’t set, no error occurs.
upgrade: If an organization reference has fewer fields than an organization, no warning is issued.

0.2.18 (2020-12-15)#

Added#

New library method:

ocdskit.util.detect_format()

0.2.17 (2020-12-11)#

Changed#

tabulate: Supports linked releases and compiled releases.

0.2.16 (2020-10-06)#

Fixed#

upgrade: If a party’s roles field isn’t a list of strings, no error occurs.

Added#

New CLI commands:

split-project-packages

0.2.15 (2020-09-30)#

Changed#

upgrade: If a party’s roles field is a string, it is coerced to an array.

Added#

New CLI options:

mapping-sheet: --no-deprecated, --no-replace-refs

0.2.14 (2020-09-16)#

Added#

New CLI option:

schema-strict: --check

New library method:

ocdskit.schema.add_validation_properties()

0.2.13 (2020-09-16)#

Fixed#

convert-to-oc4ids no longer errors if a party’s id field isn’t set.

0.2.12 (2020-08-19)#

Changed#

ocdskit.util.get_ocds_minor_version() now supports records.

Fixed#

mapping-sheet correctly populates the extension column for extension fields on OCDS objects that $ref’erence OCDS definitions.

0.2.11 (2020-08-12)#

Changed#

mapping-sheet: The extension column is now the name of the extension that introduced the JSON path, not the name of the extension that defined the field.

0.2.10 (2020-07-28)#

Changed#

indent respects --ascii.
tabulate supports any OCDS data.

Fixed#

compile no longer errors on Windows when using the SQLite backend.

0.2.9 (2020-07-08)#

Fixed#

detect-format respects --root-path.
convert-to-oc4ids omits sector and documents if empty.

0.2.8 (2020-04-29)#

Changed#

schema-strict accepts a filename as a positional argument, instead of a schema as standard input.
schema-strict adds constraints to all fields, not only required fields.

0.2.7 (2020-04-23)#

Added#

New CLI options:

combine-record-packages: --version
combine-release-packages: --version
compile: --version
convert-to-oc4ids: --version
package-records: --version
package-releases: --version

New library method:

ocdskit.util.is_compiled_release()

Changed#

compile sets "version": "1.1" even on OCDS 1.0 data (see merge()).
package-records and package-releases omit the extensions field if empty (see package_records(), package_releases()).

Fixed#

convert-to-oc4ids sets the publishedDate field, not the published_date field.

0.2.6 (2020-04-15)#

Added#

New library method:

ocdskit.util.is_linked_release()

Changed#

combine-record-packages and combine-release-packages warn if the "records" and "releases" fields aren’t set (see combine_record_packages(), combine_release_packages()).

0.2.5 (2020-04-14)#

Fixed#

combine-record-packages and combine-release-packages no longer error if the "records" and "releases" fields aren’t set (see combine_record_packages(), combine_release_packages()).

0.2.4 (2020-03-19)#

Fixed#

convert-to-oc4ids no longer errors.

0.2.3 (2020-03-19)#

Added#

New CLI commands:

convert-to-oc4ids

New library module:

ocdskit.oc4ids

Changed#

compile errors if an ocid field is missing from a release (see add_release()).
upgrade upgrades records (see upgrade_10_11()).

0.2.2 (2020-01-07)#

Changed#

Avoid exception when piping output to tools like head.
package-records, package-releases: Use fast writer if --size is set.
echo: Use fast writer (assuming --root-path is set anytime input is too large).

0.2.1 (2020-01-06)#

Added#

New CLI options:

package-records: --size
package-releases: --size

New CLI commands:

echo

Changed#

Implement iterative JSON writer.
Use orjson if available to improve performance of dumping/loading JSON, especially to/from SQL in compile command (see ocdskit.packager).

Fixed#

combine-record-packages no longer duplicates release package URLs in packages field (see ocdskit.combine.combine_record_packages()).

0.2.0 (2019-12-31)#

Added#

New library module:

ocdskit.packager

Changed#

CLI:

compile accepts either release packages or individual releases (see merge()).
compile is memory efficient if given a long list of inputs (see merge()).

Library:

Deprecate ocdskit.combine.compile_release_packages in favor of ocdskit.combine.merge().

Fixed#

--linked-releases no longer uses the same linked releases for all records (see output_records()).

0.1.5 (2019-12-18)#

Added#

New library methods:

The internal methods ocdskit.util.json_load and ocdskit.util.json_loads are removed.

0.1.4 (2019-11-28)#

Added#

New CLI options:

combine-record-packages: --fake
combine-release-packages: --fake
compile: --fake
package-records: --fake
package-releases: --fake

New CLI commands:

package-records

New library methods:

ocdskit.combine.package_records()

Changed#

mapping-sheet: Improve documentation of --extension and --extension-field.

Fixed#

detect-format correctly detects concatenated JSON, even if subsequent JSON values are non-OCDS values.

0.1.3 (2019-09-26)#

Changed#

Set missing package metadata to "" instead of null in CLI commands, to mirror library methods.

0.1.2 (2019-09-25)#

Changed#

Align the library methods ocdskit.util.json_dump() and ocdskit.util.json_dumps().

0.1.1 (2019-09-19)#

Fixed#

upgrade no longer errors if specific fields are null.
upgrade no longer errors on packages that have parties values without id fields and that declare no version or a version of “1.0”.

0.1.0 (2019-09-17)#

Command-line inputs can now be concatenated JSON or JSON arrays, not only line-delimited JSON.

Added#

New CLI commands:

detect-format

New CLI options:

package-releases:
- --uri
- --published-date
- --publisher-name
- --publisher-uri
- --publisher-scheme
- --publisher-uid
compile:
- --publisher-name
- --publisher-uri
- --publisher-scheme
- --publisher-uid
combine-record-packages:
- --publisher-name
- --publisher-uri
- --publisher-scheme
- --publisher-uid
combine-release-packages:
- --publisher-name
- --publisher-uri
- --publisher-scheme
- --publisher-uid
mapping-sheet:
- --order-by
- --infer-required
- --extension
- --extension-field

The --root-path option is added to all OCDS commands.

New library methods:

Changed#

Backwards-incompatible: upgrade_10_10(), upgrade_11_11() and upgrade_10_11() now return data, instead of only editing in-place.
Backwards-incompatible: mapping-sheet and schema-report now read a file argument instead of standard input, to support schema that $ref other schema.
mapping-sheet and schema-report support schema from: Open Contracting for Infrastructure Data Standard (OC4IDS), Beneficial Ownership Data Standard (BODS), and Social Investment Data Lab Specification (SEDL).
mapping-sheet outputs:
- enum values of items
- enum as “Enum:” instead of “Codelist:”
- pattern as “Pattern:”
schema-strict adds "uniqueItems": true to all arrays, unless --no-unique-items is set.
Use https:// instead of http:// for standard.open-contracting.org.

Fixed#

compile merges extensions’ schema into the release schema before merging releases.
mapping-sheet fills in the deprecated column if an object field uses $ref.
schema-strict no longer errors if a required field uses $ref.
upgrade no longer errors if awards or contracts is null.

0.0.5 (2019-01-11)#

Added#

New CLI options:

compile:
- --schema: You can create compiled releases and versioned releases using a specific release schema.
- --linked-releases: You can have the record package use linked releases instead of full releases.
- --uri, --published-date: You can set the uri and publishedDate of the record package.
  - If not set, these will be null instead of the uri and publishedDate of the last package.
combine-record-packages: --uri, --published-date
combine-release-packages: --uri, --published-date

New CLI commands:

upgrade

Changed#

compile raises an error if the release packages use different versions.
compile determines the version of the release schema to use if --schema isn’t set.
compile, combine-record-packages and combine-release-packages have a predictable field order.
measure is removed.

Fixed#

indent prints an error if a path doesn’t exist.
compile, combine-record-packages and combine-release-packages succeed if the required publisher field is missing.

0.0.4 (2018-11-23)#

Added#

New CLI options:

schema-report: --no-codelists, --no-definitions, --min-occurrences

Changed#

schema-report reports definitions that can use a common $ref in the versioned release schema.
schema-report reports open and closed codelists in CSV format.

0.0.3 (2018-11-01)#

Added#

New CLI options:

compile: --package, --versioned

New CLI commands:

Changed#

Add helpful error messages if:
- the input is not line-delimited JSON data.
- the input to the indent command is not valid JSON.
Change default behavior to print UTF-8 characters instead of escape sequences.
Add --ascii option to print escape sequences instead of UTF-8 characters.
Rename base exception class from ReportError to OCDSKitError.

0.0.2 (2018-03-14)#

Added#

New CLI options:

validate: --check-urls and --timeout

New CLI commands:

0.0.1 (2017-12-25)#

Added#

New CLI commands: