Skip to content

Releases: agile-lab-dev/wasp

v2.36.0

31 Jan 10:58
@tmnd1991 tmnd1991
v2.36.0
09eface
Compare
Choose a tag to compare
v2.36.0 Latest
Latest

New Features and Improvements

  • PostgreSQL Plugin with INSERT ON CONFLICT Support ([!450])

    • Added support for writing to PostgreSQL, including upserts using INSERT ON CONFLICT DO UPDATE.
    • Configurable update clauses, connection pooling, and batched writes.
    • Supports streaming query writes, allowing real-time data processing.
    • Additional enhancements include better query optimization and performance tuning.
    • Closes #460.

  • Streaming Query Watchdog ([!449])

    • Introduced a mechanism to automatically kill streaming queries exceeding a time threshold.
    • Configurable via wasp.spark-streaming.watchdog settings.
    • Added logging for better debugging and visibility into query execution.
    • Additional improvements to ensure efficient resource management.
    • Closes #568.

  • sbt Plugin Upgrades ([!438])

    • Upgraded various sbt plugins to resolve eviction errors:
      • scoverage 1.9.3
      • sbt-native-packager 1.11.0
      • sbt-buildinfo 1.13.1
      • sbt-sonatype 3.9.21
      • sbt-gpg 2.3.1
    • Fix: Corrected coverage report generation in GitLab.
    • Ensured compatibility with latest sbt versions and best practices.

  • CI and Build Improvements ([!452])

    • Upgraded sbt to 1.10.7.
    • CI image now builds cross-platform (x64, arm64).
    • Updated embedded PostgreSQL test dependency for Apple Silicon compatibility.
    • Adjusted CI execution across different architectures for better efficiency.
    • Optimized test execution times to improve build speed.
    • Removed unused flavors: CDH6.13, and EMR 6.0.
    • Closes #569.

Bug Fixes

  • JSON Parsing Mode Fix ([!446])
    • Fixed strict mode to correctly validate records matching ETL model definitions.
    • Previously, invalid records were incorrectly accepted, leading to data inconsistencies.
    • Improved error reporting for easier debugging and issue identification.
    • Added additional validation checks to prevent incorrect JSON parsing.
    • Tested via KafkaStructuredStreamingReaderSpec.
    • Closes #567.

Breaking Changes

  • Removal of CDH6.13, and EMR 6.0 flavors from the build.
    • This impacts legacy systems still using these versions, requiring migration to supported versions.
    • Developers should review dependencies to ensure compatibility.

Migration

  • No migration steps required for this release, except for systems dependent on removed flavors.
    • Users are encouraged to upgrade to supported versions for continued stability and performance.
Assets 2
Loading