You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Significantly enhance the safety of metadata manipulation
- move constants and validation logic from creator.py and constants.py
to zim/metadata.py
- add way more typing around metadata manipulation
- add new class for standard metadata, and alias for value
differentiating raw and clean metadata
- make config_metadata much stricter around standard metadata
- add config_any_metadata for those need to add custom ones or wanting
to tinker with standard ones
Copy file name to clipboardExpand all lines: CHANGELOG.md
+17Lines changed: 17 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
11
11
12
12
- Renamed `filesystem.validate_zimfile_creatable` to `filesystem.file_creatable` to reflect general applicability to check file creation beyond ZIM files #200
13
13
- Remove any "ZIM" reference in exceptions while working with files #200
14
+
- Significantly enhance the safety of metadata manipulation (#205)
15
+
- add types for all metadata, one type per metadata name plus some generic ones for non-standard metadata
16
+
- all types are responsible to validate metadata value at initialization time
17
+
- validation checks for adherence to the ZIM specification and conventions are automated
18
+
- cleanup of unwanted control characters and stripping white characters are automated
19
+
- whenever possible, try to clean a "reasonably" bad metadata (e.g. automaticall accept and remove duplicate tags - harmless - but not duplicate language codes - codes are supposed to be ordered, so it is a weird situation)
20
+
- it is now possible to disable ZIM conventions checks with `zim.metadata.check_metadata_conventions`
21
+
- simplify `zim.creator.Creator.config_metadata` by using these types and been more strict:
22
+
- add new `StandardMetadata` class for standard metadata, including list of mandatory one
23
+
- by default, all non-standard metadata must start with `X-` prefix
24
+
- this not yet an openZIM convention / specification, so it is possible to disable this check with `fail_on_missing_prefix` argument
25
+
- simplify `add_metadata`, use same metadata types
26
+
- simplify `zim.creator.Creator.start` with new types, and drop all metadata from memory after being passed to the libzim
27
+
- drop `zim.creator.convert_and_check_metadata` (not usefull anymore, simply use proper metadata type)
28
+
- move `MANDATORY_ZIM_METADATA_KEYS` and `DEFAULT_DEV_ZIM_METADATA` from `constants` to `zim.metadata` to avoid circular dependencies
29
+
- new `inputs.unique_values` utility function to compute the list of uniques values from a given list, but preserving initial list order
30
+
- in `__init__` of `zim.creator.Creator`, rename `disable_metadata_checks` to `check_metadata_conventions` for clarity and brevity
0 commit comments