Design Note: Sync Hidden

SyncHidden handles the transfer of data from servers to trading lanes in the background so that the user does not need to manually perform this task. SyncHidden is used inside fpos.dll (POS), a similar concept is used when Gds is replicating, see PeerMesh

When SyncHidden is enabled, using a normal BisSync is not disabled but modified to remove tasks that SyncHidden is performing.

In general terms, SyncHidden periodically checks with the server (Gds/2) to download and upload information. Tables are downloaded in using Gnap format files. Once downloaded a sub process is created to load data into the database which ensures the main POS does not block while loading.

Source Code: StoreServer.cpp and others

SycnHidden is controlled by the setting SyncHidden. The default value is 0 (off)

• Bit 1 (1) must be set to enable SyncHidden
• Bit 2 (2) indicates a pre scan is to be performed at startup
• Bit 3 (4) flag that table pushing is enabled (minimal support)
• Bit 4 (8) indicates the files are fetched from the server

Execution

A private thread is created (type THR_STORE_SERVER)

1. The setting SyncHiddenRandomStartDelay is fetched and used to perform a random start delay in seconds. If the setting is 0 no delay is inserted. This random pause is used to ensure that large numbers of lanes starting on automatic schedules (eg reboot at 4am) do not all hit the servers at exactly the same instant.
2. The setting MeshInputFolder is feteched and the folder created if necessary. The default folder is .\Transfer. If the setting is defined, the Windows folders must exist and will not be auto created.
3. Prechecking, if enabled
   1. Scan the input folder for *.datb files matching RmSystem etc.
   2. Check if this table is in the SyncHiddenOnlyPull or SyncHiddenNoPull settings and remove from processing if necessary. Otherwise queue the file for processing
   3. Check the setting SyncHiddenManualPull (comma seperated list) and add these tables to the processing queue.

   Main Loop

4. A task is selected from the primary work queue (preferred), or the background queue.
5. Task type specific code is invoked for the following
   • Customer Buying History
   • Db Physkey checks (verifying a record has been written to server)
   • Sale Preconfirming (verifying a sale-immediate has been stored and removing sale for Sync list)
   • Sale-immediate sending (secondary path for resending. This is not the primary send method)
   • Table fetch and Load. Pass loading command to subprocess. The command to fetch and load an individual table is

     maint(mesh(pull(...)))

   • Send table data
   • Table GET. directly pull table now from server (See PeerMesh)
6. Periodically, sales are scanned for unconfirmed sales. These are added as Sale preconfirming to the task list
7. Periodically, every SyncHiddenTableRetry seconds (default 150 minutes) tables are rescanned from the server.
8. Periodically, download the syncindex file/command and obey it's instructions.