fix(history-sync): emit completion before contact upsert#2510
Open
alexandrereyes wants to merge 2 commits intoEvolutionAPI:developfrom
Open
fix(history-sync): emit completion before contact upsert#2510alexandrereyes wants to merge 2 commits intoEvolutionAPI:developfrom
alexandrereyes wants to merge 2 commits intoEvolutionAPI:developfrom
Conversation
Contributor
Reviewer's GuideAdjusts WhatsApp Baileys history sync so the MESSAGING_HISTORY_SET completion event is emitted when sync logically finishes (before contacts.upsert), adds counters to report synced entities, introduces reliability safeguards for audio MESSAGES_UPSERT events, wires the new event into all event transport configs and validation, and adds a GitHub Actions workflow to build and publish a GHCR image. Sequence diagram for updated WhatsApp history sync completion and MESSAGING_HISTORY_SET emissionsequenceDiagram
participant Baileys as BaileysStartupService
participant BaileysCore as BaileysHistorySync
participant Store as PrismaRepository
participant Events as EventsDispatcher
participant Contacts as ContactsHandler
BaileysCore->>Baileys: historySyncHandler(messages, chats, contacts, progress)
alt progress reset or decrease
Baileys->>Baileys: historySyncMessageCount = 0
Baileys->>Baileys: historySyncChatCount = 0
Baileys->>Baileys: historySyncContactCount = 0
end
Baileys->>Baileys: historySyncLastProgress = progress
Baileys->>Store: chat.createMany(chatsRaw)
Baileys->>Baileys: historySyncChatCount += chatsRaw.length
Baileys->>Events: sendDataWebhook(CHATS_SET, chatsRaw)
Baileys->>Store: message.createMany(messagesRaw)
Baileys->>Baileys: historySyncMessageCount += messagesRaw.length
Baileys->>Events: sendDataWebhook(MESSAGES_SET, messagesRaw, isLatest, progress)
Baileys->>Baileys: filteredContacts = contacts with notify or name
Baileys->>Baileys: historySyncContactCount += filteredContacts.length
alt progress == 100
Baileys->>Events: sendDataWebhook(MESSAGING_HISTORY_SET, {messageCount, chatCount, contactCount})
Baileys->>Baileys: reset historySyncMessageCount, historySyncChatCount, historySyncContactCount, historySyncLastProgress
end
Baileys->>Contacts: contacts.upsert(filteredContacts)
Contacts-->>Baileys: upsert completed
Sequence diagram for audio MESSAGES_UPSERT emission with fallbacksequenceDiagram
participant Baileys as BaileysStartupService
participant Cache as BaileysCache
participant Events as EventsDispatcher
participant Chatbot as ChatbotController
rect rgb(235, 245, 255)
Note over Baileys: Initial audio message upsert
Baileys->>Events: sendDataWebhook(MESSAGES_UPSERT, messageRaw)
alt audioMessage and not fromMe and key.id exists
Baileys->>Cache: set(upsert_emitted_instanceId_messageId, true, ttl=600)
end
Baileys->>Chatbot: emit(messageRaw)
end
rect rgb(245, 235, 255)
Note over Baileys: Later message update for same audio message
Baileys->>Baileys: handleMessagesUpdate(key)
alt not fromMe and findMessage.messageType == audioMessage and key.id exists
Baileys->>Cache: get(upsert_emitted_instanceId_messageId)
alt alreadyEmitted == false
Baileys->>Events: sendDataWebhook(MESSAGES_UPSERT, fallbackUpsertPayload)
Baileys->>Cache: set(upsert_emitted_instanceId_messageId, true, ttl=600)
end
end
end
Class diagram for updated BaileysStartupService history sync and audio upsert handlingclassDiagram
class BaileysStartupService {
- logBaileys Log
- eventProcessingQueue Promise_void
- historySyncMessageCount number
- historySyncChatCount number
- historySyncContactCount number
- historySyncLastProgress number
- MESSAGE_CACHE_TTL_SECONDS number
- UPDATE_CACHE_TTL_SECONDS number
+ historySyncHandler(messages any[], chats any[], contacts any[], progress number, isLatest boolean, syncType HistorySyncType) Promise_void
- getUpsertEmittedCacheKey(messageId string) string
+ handleIncomingMessage(received any) Promise_void
+ handleMessagesUpdate(key any) Promise_void
+ sendDataWebhook(event Events, payload any, isArray boolean, headers any, meta any) Promise_void
}
class BaileysCache {
+ get(key string) Promise_any
+ set(key string, value any, ttlSeconds number) Promise_void
}
class ContactsHandler {
+ contacts_upsert(contacts ContactUpsertDto[]) Promise_void
}
class EventsDispatcher {
+ dispatchCHATS_SET(chats any[]) Promise_void
+ dispatchMESSAGES_SET(messages any[], isLatest boolean, progress number) Promise_void
+ dispatchMESSAGES_UPSERT(message any) Promise_void
+ dispatchMESSAGING_HISTORY_SET(summary HistorySyncSummary) Promise_void
}
class PrismaRepository {
+ chat_createMany(data any[], skipDuplicates boolean) Promise_void
+ message_createMany(data any[], skipDuplicates boolean) Promise_void
+ media_create(data any) Promise_void
+ message_update(where any, data any) Promise_void
}
BaileysStartupService --> BaileysCache : uses
BaileysStartupService --> ContactsHandler : uses
BaileysStartupService --> EventsDispatcher : uses
BaileysStartupService --> PrismaRepository : uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- Consider extracting the
60 * 10TTL used for the upsert-emitted cache into a named constant (similar to the other cache TTL constants at the top of the class) to avoid magic numbers and make future adjustments easier. - The
historySyncLastProgresshandling currently relies on-1as a sentinel andprogress ?? -1; you may want to make this explicitlynumber | null(and handleundefined/nullin the reset logic) to make the intent clearer and avoid subtle issues ifprogressis ever missing or non-numeric.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider extracting the `60 * 10` TTL used for the upsert-emitted cache into a named constant (similar to the other cache TTL constants at the top of the class) to avoid magic numbers and make future adjustments easier.
- The `historySyncLastProgress` handling currently relies on `-1` as a sentinel and `progress ?? -1`; you may want to make this explicitly `number | null` (and handle `undefined`/`null` in the reset logic) to make the intent clearer and avoid subtle issues if `progress` is ever missing or non-numeric.
## Individual Comments
### Comment 1
<location path="src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts" line_range="953-959" />
<code_context>
syncType?: proto.HistorySync.HistorySyncType;
}) => {
try {
+ // Reset counters when a new sync starts (progress resets or decreases)
+ if (progress <= this.historySyncLastProgress) {
+ this.historySyncMessageCount = 0;
+ this.historySyncChatCount = 0;
+ this.historySyncContactCount = 0;
+ }
+ this.historySyncLastProgress = progress ?? -1;
+
if (syncType === proto.HistorySync.HistorySyncType.ON_DEMAND) {
</code_context>
<issue_to_address>
**issue (bug_risk):** History sync counters can leak between sync sessions when `progress` is undefined.
The reset check runs before `this.historySyncLastProgress = progress ?? -1;`. If a new sync starts with `progress` undefined and `historySyncLastProgress` still at, say, 100 from the previous sync, `undefined <= 100` is false, so counters are not reset, and then `historySyncLastProgress` becomes `-1`. The first chunk of the new sync is added to the old counters, and only a later event with defined `progress` will reset them.
To avoid this, normalize `progress` first and use the normalized value for both comparison and assignment, e.g.:
```ts
const normalizedProgress = progress ?? -1;
if (normalizedProgress <= this.historySyncLastProgress) {
this.historySyncMessageCount = 0;
this.historySyncChatCount = 0;
this.historySyncContactCount = 0;
}
this.historySyncLastProgress = normalizedProgress;
```
Alternatively, always reset when `progress` is `undefined` and treat that as the start of a new sync.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+953
to
+959
| // Reset counters when a new sync starts (progress resets or decreases) | ||
| if (progress <= this.historySyncLastProgress) { | ||
| this.historySyncMessageCount = 0; | ||
| this.historySyncChatCount = 0; | ||
| this.historySyncContactCount = 0; | ||
| } | ||
| this.historySyncLastProgress = progress ?? -1; |
Contributor
There was a problem hiding this comment.
issue (bug_risk): History sync counters can leak between sync sessions when progress is undefined.
The reset check runs before this.historySyncLastProgress = progress ?? -1;. If a new sync starts with progress undefined and historySyncLastProgress still at, say, 100 from the previous sync, undefined <= 100 is false, so counters are not reset, and then historySyncLastProgress becomes -1. The first chunk of the new sync is added to the old counters, and only a later event with defined progress will reset them.
To avoid this, normalize progress first and use the normalized value for both comparison and assignment, e.g.:
const normalizedProgress = progress ?? -1;
if (normalizedProgress <= this.historySyncLastProgress) {
this.historySyncMessageCount = 0;
this.historySyncChatCount = 0;
this.historySyncContactCount = 0;
}
this.historySyncLastProgress = normalizedProgress;Alternatively, always reset when progress is undefined and treat that as the start of a new sync.
Author
|
@sourcery-ai review |
Contributor
|
Sorry @alexandrereyes, you have reached your weekly rate limit of 500000 diff characters. Please try again later or upgrade to continue using Sourcery |
1 similar comment
|
Sorry @alexandrereyes, you have reached your weekly rate limit of 500000 diff characters. Please try again later or upgrade to continue using Sourcery |
alexandrereyes
force-pushed
the
fix-history-sync-completion-event
branch
from
10f1cee to
86b0c44
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The history sync flow can fail or stall after the final
MESSAGING_HISTORY_SETshould have been emitted. In the previous order, the completion event was sent only aftercontacts.upsert, so when that later step failed or took too long, consumers could miss the final completion signal even though the sync had already reached the completion point.Solution
MESSAGING_HISTORY_SETemission to run beforecontacts.upsertin the history sync handlerprogressbefore comparing it to the last observed sync progress, so counters reset correctly when a new sync starts without a numeric progress valueMESSAGING_HISTORY_SET🧪 Type of Change
🧪 Testing
npm run buildnpm run lint:check✅ Checklist