Services

RabbitMQ

See also

• Worker design

• Having trouble? See Troubleshoot.

This project uses a direct exchange in the same way as the default exchange, in which every queue is bound using a routing key that is the same as the queue’s name.

In each worker, the queue name and binding key is set by consume_routing_key. The routing key of published messages is set by routing_key. Queue names and routing keys are prefixed by the exchange name, set by the RABBIT_EXCHANGE_NAME environment variable.

Worker/Command	Queue (input)	Message routing key (output)	Sample message
manage.py add	N/A	ocds_kingfisher_extractor_init	add command
workers.extract.kingfisher_process	ocds_kingfisher_extractor_init	extractor	{"dataset_id": 1, "item_ids": [1,2,3]}
workers.extract.dataset_filter	dataset_filter_extractor_init	extractor	{"dataset_id": 1, "item_ids": [1,2,3]}
workers.check.data_item	extractor	contracting_process_checker	{"dataset_id": 123}
workers.check.dataset	contracting_process_checker	dataset_checker	{"dataset_id": 123}
workers.check.time_based	dataset_checker	time_variance_checker	{"dataset_id": 123}
workers.report	time_variance_checker	N/A	N/A
workers.wipe	wiper_init	N/A	N/A

PostgreSQL

Tables

Table	Description	INSERT by	SELECT by
dataset	Extracted collections	workers.extract	workers.extract.dataset_filter, dataset/meta_data_aggregator, time_variance/processor
data_item	Extracted compiled releases	workers.extract	workers.check.data_item, dataset/processor, time_variance/processor
field_level_check	Field-level check results	contracting_process/processor	contracting_process/field_level/report_examples
field_level_check_examples	Field-level check examples	contracting_process/processor	contracting_process/field_level/report_examples
resource_level_check	Compiled release-level check results	contracting_process/processor	contracting_process/resource_level/examples, contracting_process/resource_level/report
resource_level_check_examples	Compiled release-level check examples	contracting_process/processor	contracting_process/resource_level/examples
dataset_level_check	Dataset-level check results	dataset/processor	N/A
time_variance_level_check	Time-based check results	time_variance/processor	N/A
progress_monitor_dataset	Progress of datasets’ processing (see State machine)	workers.extract, workers.check, workers.report	workers.extract.dataset_filter, workers.check.dataset
progress_monitor_item	Progress of items’ processing (see State machine)	workers.extract, contracting_process/processor	workers.check.dataset