Basic Information

  • SSEC Satellite Data Services (SDS) is using RabbitMQ servers to provide events to users signaling when real-time and archive data become available.
  • The utility python script amqpfind creates connections to the RabbitMQ servers and listens for events published by SDS ingest processes.
  • Three different methods of listening for events. Example code and commands can be found at the following:


You can download from the SDS website. The package also contains examples using the amqpfind library to customize scripts.

Python requirements

Multi-server: Python 2.7 and greater
Single Server: Python 2.6 and greater

Tested operating systems

Redhat/CentOS 6 and 7
OS X 10.0


amqpfind -H -u -p [-t] [-X] [-j'{key} {key}' | '{?}'] [-C key | key]

-H host ip address of the Satellite Data Services RabbitMQ server mq1(2)
-u user for SDS RabbitMQ server (sdsuser)
-p password for SDS RabbitMQ server (sdsmq)
-j outputs subset of json key values of the message; must be a single quoted string, each key must be surrounded by curly braces; for example: -j '{start_time} {band} {status}'
-j '?' lists all keys and values from message, curly brackets surrounding question mark may be necessary for some shell types
-k (multi-server only) keys checked to see is a message, sent from multiple servers, is considered a duplicate
-w (multi-server only) time window, in second, used to watch for duplicate messages
-t sets timeout in seconds before amqpfind ends
-X SDS RabbitMQ server exchange; valid options are satellite or himawari
-C period-separated list of nine subscription keys; specific values or wild card must be specified for all nine keys; options shown below
-v debug level

Subscription keys (-C)

GOES-R Series
g16 or g17
abi glm exis magnetometer seiss suvi
file or adde file or adde file file file file
realtime, archive or sdi archive or adde realtime realtime realtime realtime
band, image or product point image image image image
start, end or complete complete complete complete complete complete
h8 or 8; h9 or 9
file or adde
satbuf, arcdata or cluster
granule, band, section or image
n20 or snpp
band or image
end or complete

 Basic Examples

Outputs a user friendly list all messages and content when a GOES-R series satellite abi instrument completes an image:

amqpfind -H -u sdsuser -p sdsmq -X satellite -j '?' -C geo.goes.*.abi.*.*.*.image.complete

The intended use of amqpfind is subscribe to one or more message topics and start a script using xargs:

amqpfind -H -u sdsuser -p sdsmq -j '{adde_dataset} {start_time}' \
-X satellite -C geo.goes.g16.abi.adde.sdi.ncdf.image.complete | xargs -P1 -n3 myscript.bash

The command started looks as follows:

myscript.bash EASTA/M1 2018-01-23 21:54:29.8 8

In the above xargs command, -P1 limits xargs to use a single processor. If your processing script takes a significant amount of time to complete, you may need to increase the number of processes available to xargs. As an example, the above amqpfind command returns an message for all GOES-16 images, if your processing for a FD image takes longer than 30 seconds, Mesoscale images must wait until the FD processing completes.

Detailed Information

The RabbitMQ utility amqpfind runs in either single server or multi-server mode and makes the necessary connections to RabbitMQ server(s) allowing clients to listen for event messages (e.g., image complete or band end).  When a client finds a specific event, a script can be launched and use a message's metadata as input. Metadata in the messages include values such as image start time, coverage, band, server IP and much more. All events may not be of interest so a key based subscription can filter out unwanted events. SDS ingestors publishe message to multiple RabbitMQ, so in the case of one RabbitMQ server failing a user has a backup source of events. SDS maintains the key names and message content. As new uses, new instruments and new facilities are rolled out, expect incremental changes in both message types and content. The amqpfind utility provides basic connections to the SDS RabbitMQ servers using the Advanced Message Queuing Protocol (AMQP). Custom scripts using this protocol or others can be used to subscribe to the SDS RabbitMQ servers. At no time will users be allowed to publish messages to the SDS RabbitMQ servers.