Changed system-accounts/initialize and pool-accounts/initialize endpoints from
@AdminOnly to @Public to allow deploy scripts to call them without authentication.
These endpoints are only accessible from internal network.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use admin.resetOffsets({ earliest: true }) before connecting consumer
to ensure CDC sync always starts from the beginning of Kafka topics,
regardless of previously committed offsets.
This fixes the infinite loop issue where existing consumer groups
had committed offsets at high watermark, causing eachMessage to
never be called.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change CDC consumer startup from blocking await to non-blocking .then()
so HTTP server starts immediately and /health/cdc-sync API is accessible
for deploy script to poll sync status.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change await to .then() for cdcConsumer.start()
- Allows HTTP endpoints to be accessible during CDC sync
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add initialSyncCompleted flag to track CDC sequential sync completion
- Add getSyncStatus() method to CDCConsumerService
- Add /health/cdc-sync endpoint to expose sync status
- Update deploy-mining.sh to wait for CDC sync completion before calling publish APIs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previous consumer group had already consumed messages, so fromBeginning
had no effect. Now using timestamp-based unique group id to ensure
fresh consumption from beginning each time.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous change was wrong - running sequential consumption in
background defeats its purpose. The whole point is to ensure data
dependency order (users -> referrals -> adoptions) before any other
operations can proceed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Prevents blocking NestJS onModuleInit during CDC sync by running
the sequential consumption in the background with error handling.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements sequential phase consumption to ensure correct data sync order:
1. User accounts (first)
2. Referral relationships (depends on users)
3. Planting orders (depends on users and referrals)
Each phase must complete before the next starts, guaranteeing 100%
reliable data dependency ordering. After all phases complete, switches
to continuous parallel consumption for real-time updates.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Reverts commits:
- 1fbb88f7: setDirectReferralAdoptedCount change
- 471702d5: ancestor_path chain building change
These changes were made without authorization. The original code was correct.
MINING_ENABLED filtering (from dbf97ae4) is preserved.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changed updateReferrerUnlockStatus to:
1. Create account if not exists (for full-reset scenarios)
2. Use setDirectReferralAdoptedCount instead of increment loop
3. This ensures the count is always accurate regardless of processing order
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Users were being redirected to login page when clicking navigation
because the background token refresh was failing and clearing user state.
Token refresh should only happen when API returns 401, not on every app launch.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add system account types and display metadata
- Create API layer with getList and getSummary endpoints
- Add React Query hooks for data fetching
- Create AccountCard, AccountsTable, SummaryCards components
- Refactor page with tabs, refresh button, and error handling
- Add Alert UI component
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: CDC sync order issue caused referrerAccountSequence to be null,
resulting in empty ancestor chain and all TEAM_LEVEL contributions going to unallocated.
Changes:
- buildAncestorChainFromReferral: Uses ancestor_path (contains complete user_id chain) to build upline chain
- getDirectReferrer: Gets direct referrer using ancestor_path as fallback
- findAncestorChain: Updated to use ancestor_path when available
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Only process adoptions with MINING_ENABLED status for contribution calculation.
This fixes the bug where non-final adoption records (PENDING, PAID, etc.) were
incorrectly being processed, causing duplicate contribution records.
Affected methods:
- findUndistributedAdoptions: only process MINING_ENABLED adoptions
- getDirectReferralAdoptedCount: only count users with MINING_ENABLED adoptions
- getTotalTreesByAccountSequence: only sum trees from MINING_ENABLED adoptions
- getTeamTreesByLevel: only count MINING_ENABLED adoptions
- countUndistributedAdoptions: only count MINING_ENABLED adoptions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ContributionAccountUpdatedEvent for real-time account updates
- Publish outbox events when saving distribution results
- Publish outbox events when updating adopter/referrer unlock status
- Add incremental sync every 10 minutes for recently updated accounts
- Add daily full sync at 4am as final consistency guarantee
- Add findRecentlyUpdated repository method for incremental sync
Three-layer sync guarantee:
1. Real-time: publish events on every account update
2. Incremental: scan accounts updated in last 15 minutes every 10 mins
3. Full sync: publish all accounts daily at 4am
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove conditional skip logic in CDC handlers
- Always sync all field updates (including status changes)
- Trigger contribution calculation only when status becomes MINING_ENABLED
- Fix user and referral handlers to sync all fields without skipping
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Revert the previous change that removed the status filter. The stats
should only count adoptions with MINING_ENABLED status, as only those
are active for mining. The issue is likely that the status field in
synced_adoptions table doesn't have the correct value.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The adoption stats were showing 0 because the synced_adoptions table
contains status values directly from 1.0 system (PAID, POOL_INJECTED, etc.)
rather than MINING_ENABLED. Since contribution-service doesn't update the
status after calculating contributions, we now count all synced adoptions.
Changes:
- Remove status filter in getAdoptionStatsForUsers
- Remove status filter in getUserDetail adoption queries
- Remove status filter in getUserAdoptionStats for referral tree
- Add order count display in user detail page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Backend:
- Add personalOrders and teamOrders to adoption stats
- Return order count alongside tree count in user list API
Frontend:
- Add personalAdoptionOrders and teamAdoptionOrders to UserOverview type
- Display format: "树数量(订单数)" e.g. "6(3单)"
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Only count adoptions with status='MINING_ENABLED' when calculating:
- Personal adoption count (user list)
- Team adoption count (user list)
- Personal adoption stats (user detail)
- Direct referral adoptions (user detail)
- Team adoptions (user detail)
- Referral tree adoption stats
This fixes incorrect adoption counts that included pending/unconfirmed orders.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix hydration race condition where token check happened before
localStorage was read. Now waits for client-side initialization
before deciding whether to redirect to login.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
kafkacat/kcat not available in containers. Switch to kafka-console-producer
with null.marker property to send tombstone messages for Debezium offset deletion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Debezium Connect container uses OFFSET_STORAGE_TOPIC=debezium_offsets,
not the default connect-offsets. This fix updates the tombstone method
to use the correct topic name.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This fixes an issue where Debezium would skip initial snapshot after
full-reset because the connector offset persisted in Kafka Connect's
internal connect-offsets topic.
The fix adds two strategies to delete connector offsets:
1. REST API method (Kafka Connect 3.6+): DELETE /connectors/<name>/offsets
2. Tombstone method (Kafka Connect 3.5-): Send NULL messages via kafkacat
Reference: https://debezium.io/documentation/faq/#how_to_remove_committed_offsets_for_a_connector
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Old messages in Kafka topics were corrupting the sync because they contained
outdated data from previous resets. Now we delete the outbox topics during
full-reset to ensure clean re-sync.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The full-reset script was missing the cleanup of rwa_mining_admin.processed_events
table, which caused stale idempotency records to prevent re-consumption of
contribution outbox events after reset.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace skeleton blocks with shimmer text placeholders across all pages:
- Asset page: show full UI with flickering numbers during load
- Trading page: show full UI with flickering market data during load
- Contribution page: show full UI with flickering stats during load
- shimmer_loading.dart: add ShimmerText, DataText, AmountText components
This approach shows the complete UI immediately, with only the dynamic
number values flickering while data loads - much better UX than showing
grey skeleton blocks.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Optimize splash page: reduce wait to 500ms, refresh token in background
- Cache SharedPreferences instance to avoid blocking API requests
- Add global 401 handler to auto-redirect to login page
- Create shimmer loading components (ShimmerLoading, ShimmerBox, skeletons)
- Replace CircularProgressIndicator with skeleton screens across all pages
- Add keepAlive + auto-invalidation (5min) to providers to reduce API calls
- Fix trading page: invalidate globalStateProvider after trade for data sync
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The outbox_events table uses outbox_id as the primary key column name
(mapped from id in Prisma). When Debezium captures changes, the message
contains outbox_id field, not id. This caused all events to have
undefined eventId, resulting in duplicate detection treating all events
as duplicates after the first one.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: calculateForAdoption uses separate DB connections, which
cannot see uncommitted data in Serializable isolation level, causing
"Adoption not found" errors.
Solution (following Kafka Idempotent Consumer best practice):
- Add TransactionalCDCHandlerWithResult<T> type for handlers with return
- Add withIdempotencyAndCallback() wrapper for post-commit callbacks
- Add registerTransactionalHandlerWithCallback() registration method
- AdoptionSyncedHandler.handle() now returns AdoptionSyncResult
- Contribution calculation runs AFTER transaction commits via callback
Reference: Lydtech Consulting - Kafka Idempotent Consumer Pattern
https://www.lydtechconsulting.com/blog/kafka-idempotent-consumer-transactional-outbox
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Problem: full-reset script resets Kafka consumer offsets but doesn't
clear the processed_cdc_events table. When migrations run, containers
may start and consume CDC events, inserting records into this table.
After offset reset, the service restarts and detects all events as
"duplicates" because the idempotency records still exist.
Solution: Add TRUNCATE processed_cdc_events step after CDC offset reset
in Step 8, before starting services. This ensures the idempotency table
is in sync with the Kafka offset position.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The bcrypt package requires native compilation. Added build dependencies
to the builder stage so npm ci can compile bcrypt successfully.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add comments explaining unique key composition:
- CDC events: (source_topic, offset) = Kafka topic + message offset
- Outbox events: (source_service, event_id) = service name + outbox ID
- Fix contribution-service migration:
- Extend source_service column from VARCHAR(50) to VARCHAR(100)
- Set source_service as NOT NULL to match schema
- Use snake_case for index name consistency
- Clarify that offset/event_id are NOT database auto-increment IDs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update splash_page.dart to orange theme (#FF6B00) matching other pages
- Change app name from "榴莲挖矿" to "榴莲生态"
- Fix refreshTokenIfNeeded to properly throw on failure instead of
silently calling logout (which caused Riverpod ref errors)
- Clear local storage directly on refresh failure without remote API call
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements 100% exactly-once semantics for CDC events from 1.0 identity-service
(user_accounts table) to auth-service.
Key changes:
- Add ProcessedCdcEvent model with (sourceTopic, offset) unique constraint
- Implement processWithIdempotency() using Serializable transaction isolation
- All database operations now use the transaction client
- Outbox event creation is also within the same transaction
This ensures that:
1. Each CDC event is processed exactly once
2. Idempotency record and business logic are in the same transaction
3. Outbox event publishing is atomic with data sync
4. Any failure causes complete rollback
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements 100% exactly-once semantics for CDC events from 1.0 databases
(identity-service, planting-service, referral-service) to contribution-service.
Key changes:
- Add ProcessedCdcEvent model with (sourceTopic, offset) unique constraint
- Add withIdempotency() wrapper using Serializable transaction isolation
- Add registerTransactionalHandler() for handlers requiring idempotency
- Modify CDC handlers to accept external transaction client
- All database operations now use the passed transaction client
This ensures that:
1. Each CDC event is processed exactly once
2. Idempotency record and business logic are in the same transaction
3. Any failure causes complete rollback
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use Prisma $transaction with Serializable isolation level
- Insert idempotency record FIRST, then execute business logic
- Unique constraint violation (P2002) indicates duplicate event
- All operations atomic - either fully commit or fully rollback
- Modified all handlers to accept transaction client parameter
- Removed old non-atomic isEventProcessed/recordProcessedEvent methods
This ensures 100% data consistency for CDC synchronization, which is
critical for financial data where any error is catastrophic.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use (sourceTopic, eventId) as composite unique key in processed_events
- Add sourceTopic to ServiceEvent for globally unique idempotency key
- Wrap all handlers with withIdempotency() for duplicate event detection
- Fix ID collision issue between different service outbox tables
This implements the industry-standard CDC exactly-once semantics pattern.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Wrap content in LayoutBuilder + ConstrainedBox to ensure proper
scrolling behavior when content exceeds viewport height.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changes:
1. Delete Debezium connectors BEFORE dropping databases (releases replication slots)
2. Start services BEFORE registering connectors (ensures tables exist and data is synced)
3. Register connectors AFTER services sync from 1.0 CDC (snapshot.mode=initial captures existing data)
4. Add wait time for connectors to initialize before publishing data
Step order: stop services → delete connectors → drop DBs → create DBs → migrate →
start services → wait for sync → register connectors → publish data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update login/register pages to use orange color scheme (#FF6B00)
matching the navigation pages design
- Fix SafeArea bottom: false on all navigation pages since MainShell
handles bottom safe area via bottomNavigationBar
- Add AlwaysScrollableScrollPhysics to asset page for consistent scroll
- Increase bottom padding to 100px on all navigation pages to clear
the navigation bar
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The full-reset function was calling contribution-records/publish-all API
which caused duplicate records in mining-admin-service because:
- Contribution records are already published to outbox when calculated
- Debezium automatically captures outbox_events and sends to Kafka
- Calling publish-all again creates duplicate events with different IDs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The toDto method was accessing non-existent properties on ContributionAccountAggregate:
- teamLevelContribution -> totalLevelPending
- teamBonusContribution -> totalBonusPending
- totalContribution -> effectiveContribution
- isCalculated -> true (always calculated when account exists)
- lastCalculatedAt -> updatedAt
This was causing "Cannot read properties of undefined (reading 'value')" error
on GET /api/v2/contribution/accounts/{accountSequence}
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete old green-themed HomePage and its widgets (asset_card, price_card, quick_actions)
- Remove /home route from router configuration
- Fix SplashPage to redirect to /contribution instead of /home after login
- Now all navigation goes through the new orange-themed UI pages
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
hailin