Skip to content

Conversation

@elaine-mattos
Copy link
Contributor

Hi,

as part of the Node.js and package updates effort (#1340), here is the follow-up PR that adds backward compatibility to handle Redis cache entries compressed with both pako 1.x (binary string) and pako 2.x (base64) formats, preventing cache invalidation during the upgrade.

Summary

Adds backward compatibility to handle Redis cache entries compressed with both pako 1.x (binary string) and pako 2.x (base64) formats, preventing cache invalidation during the upgrade.

Changes

  • Format detection in redis.js: Automatically detects whether cached data is in old binary string format (pako 1.x) or new base64 format (pako 2.x)
  • Backward compatibility: Existing cache entries remain readable after upgrading to pako 2.x
  • Test coverage: Added comprehensive tests for both def_* (definitions) and hrv_* (harvest) key patterns

Testing

  • Unit tests verify reading both old (pako 1.x binary) and new (pako 2.x base64) formats
  • Tested with local Redis container using keys written with pako 1.x and read with pako 2.x
  • Tests use both pako@1.0.8 and pako@2.1.0 to prove cross-version compatibility

Benefits

  • Zero cache invalidation on deployment
  • Seamless migration from pako 1.x to 2.x
  • Both formats coexist during transition period

Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>
Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>
Signed-off-by: ElaineDeMattosSilvaB <elaine.de-mattos-silva-bezerra@deutschebahn.com>
service: config.get('CACHING_REDIS_SERVICE'),
apiKey: config.get('CACHING_REDIS_API_KEY'),
port: Number(config.get('CACHING_REDIS_PORT')) || 6380
port: Number(config.get('CACHING_REDIS_PORT'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was the removal of || 6380 here intentional? redis.js has it in the function signature default, and config.js has added the fallback.

Comment on lines +127 to +128
"pako": "^2.1.0",
"pako-1": "npm:pako@^1.0.8"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: sort alphabetically.

Copy link
Collaborator

@qtomlinson qtomlinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the migration and the unit tests!

"testcontainers": "^11.2.1",
"typescript": "5.8.3"
"typescript": "5.8.3",
"pako": "^2.1.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate? pako is already listed in dependencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it's pako v1 and v2 for backwards compat.


// Detect format: base64 (new) vs binary string (old)
// Base64 only contains A-Z, a-z, 0-9, +, /, and optional = padding
const isBase64 = /^[A-Za-z0-9+/]+=*$/.test(dataString)
Copy link
Collaborator

@qtomlinson qtomlinson Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of our cached values are objects and this works well for those cases. When a number is cached (e.g. statusService.requestCount), isBase64 becomes false positive. The number of such cases is small.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants