Conversation
pom.xml
Outdated
| <groupId>com.uid2</groupId> | ||
| <artifactId>uid2-optout</artifactId> | ||
| <version>4.9.0</version> | ||
| <version>4.9.1-alpha-215-SNAPSHOT</version> |
There was a problem hiding this comment.
will revert before merge
| } | ||
|
|
||
| PartnerConfigMonitorV2 configMon = new PartnerConfigMonitorV2(vertx, config, fsMetadata, fsContent, eventCloudSyncDownloaded); | ||
| PartnerConfigMonitorV2 configMon = new PartnerConfigMonitorV2(vertx, config, fsMetadata, fsContent, eventCloudSyncDownloaded, fsOptOut); |
There was a problem hiding this comment.
curious, do we still need 2 versions of createPartnerConfigMonitor and PartnerConfigMonitor?
There was a problem hiding this comment.
No, only v2 is used since we retrieve from core. I will create a followup to remove the old class and config version.
| this.lastProcessedTimestamp = readTimestampFromCloud(); | ||
| this.lastEntrySent.set(this.lastProcessedTimestamp.getEpochSecond()); | ||
|
|
||
| this.processedDeltas.clear(); |
There was a problem hiding this comment.
Curious, do we need the clear here, would it ever not be empty?
It implies if processedDeltas contains entries that are not in readProcessedDeltasFromCloud, we want to remove those.
There was a problem hiding this comment.
Good catch, there is no reload/rescan so this is unnecessary and misleading, will remove.
…BTechLab/uid2-optout into ian-UID2-6392-move-sender-state-to-s3
| // Step 3. purge (deleting empty deltas) | ||
| // Step 4. sync (sync again after delete) | ||
| // An OptOut CronJob performs the following tasks sequentially: | ||
| // Step 1. gc (delete expired delta/partition files from S3) |
There was a problem hiding this comment.
I don't think we delete partition files any more
| // Step 7. sync (sync again after compact) | ||
| // Step 3. pack (compact deltas into partition) | ||
| // Step 4. push (upload partition, delete source deltas from S3) | ||
| // Step 5. sync (sync again after compact) |
There was a problem hiding this comment.
What's the purpose of step 5 if the pod is going to be stateless?
There was a problem hiding this comment.
removing 5, also moving step 2 to only if -pack
| // returning name of the file that stores timestamp | ||
| public Path getTimestampFile() { | ||
| return this.timestampFile; | ||
| String getTimestampKey() { |
There was a problem hiding this comment.
Doesn't appear to be used
| // returning name of the file that stores processed deltas | ||
| public Path getProcessedDeltasFile() { | ||
| return this.processedDeltasFile; | ||
| String getProcessedDeltasKey() { |
There was a problem hiding this comment.
Doesn't appear to be used
4 participants
OptOutLogTool.java: remove the empty-delta-file removal step from the cron job, since we no longer generate empty delta files.
The rest of the changes are to move the partner sender state from local storage into S3 storage. This includes a timestamp.txt file for each partner, which tracks the timestamp of the most recent delta sent to the partner, and a processed.txt file that lists the delta files already sent to the partner.