TEM Architecture
TEM Architecture

TEM Architecture

Welcome to the documentation for the Allen Institute next generation TEM data acquisition system. This is an event driven system with various components running in docker containers, along with other services running on other machines. Below, is a diagram showing a general overview of this system.

%3 cluster_AWS AWS cluster_Docker Compose Docker Compose cluster_Platform9 Platform9 cluster_Lab Server Lab Server dddb4ffa8931499f99883efd6792b617 Database 3a5bbe3a4f3d42b3929c5f8cd9d5b753 S3 Bucket 1cf1b8ed858049bfb7c902065a3af0d7 AC/QC 3a5bbe3a4f3d42b3929c5f8cd9d5b753->1cf1b8ed858049bfb7c902065a3af0d7 10d60c338120437290b38c250dd3f665 Aloha 3a5bbe3a4f3d42b3929c5f8cd9d5b753->10d60c338120437290b38c250dd3f665 e59fc07e30ee46c79c13ed5d07461a2f TEM DB e59fc07e30ee46c79c13ed5d07461a2f->dddb4ffa8931499f99883efd6792b617 e59fc07e30ee46c79c13ed5d07461a2f->1cf1b8ed858049bfb7c902065a3af0d7 ea6d3fe06ffb45ca8c08fc1d882e038f pyTEM e59fc07e30ee46c79c13ed5d07461a2f->ea6d3fe06ffb45ca8c08fc1d882e038f 10d60c338120437290b38c250dd3f665->e59fc07e30ee46c79c13ed5d07461a2f e66a3156a78a459cb943664fa2e76d93 AC/QC user e66a3156a78a459cb943664fa2e76d93->1cf1b8ed858049bfb7c902065a3af0d7 a5cb8f6fbc4b44d586411381f0c6d5ea Microscope Operator 27d57c01f680457c85deaaf081399760 UI Server a5cb8f6fbc4b44d586411381f0c6d5ea->27d57c01f680457c85deaaf081399760 ee7a4d0d85504259a11c05a65154d7bd ActiveMQ ba54ee0a3a3c49a3811c2b54eec37075 Microscope Service ea6d3fe06ffb45ca8c08fc1d882e038f->ba54ee0a3a3c49a3811c2b54eec37075 de852d6bb54e49848fce4d1629a98a74 Camera Service ea6d3fe06ffb45ca8c08fc1d882e038f->de852d6bb54e49848fce4d1629a98a74 598343b412db4d5eb5fffdbd7c7f4e7e Stage Service ea6d3fe06ffb45ca8c08fc1d882e038f->598343b412db4d5eb5fffdbd7c7f4e7e 31f0b2edf1a1443599ae3c7d406ae275 Image Processing Pipeline ea6d3fe06ffb45ca8c08fc1d882e038f->31f0b2edf1a1443599ae3c7d406ae275 ea6d3fe06ffb45ca8c08fc1d882e038f->27d57c01f680457c85deaaf081399760 32242c0762fd4d858fa7c380d21fa869 Buffer Service 31f0b2edf1a1443599ae3c7d406ae275->32242c0762fd4d858fa7c380d21fa869 31f0b2edf1a1443599ae3c7d406ae275->27d57c01f680457c85deaaf081399760 1dff5e0a66ce49dcb30b41d77de447f7 QC Service 31f0b2edf1a1443599ae3c7d406ae275->1dff5e0a66ce49dcb30b41d77de447f7 32242c0762fd4d858fa7c380d21fa869->10d60c338120437290b38c250dd3f665 1dff5e0a66ce49dcb30b41d77de447f7->ea6d3fe06ffb45ca8c08fc1d882e038f 73c240006c4e441a9b60961ff86bb989 Microscope 73c240006c4e441a9b60961ff86bb989->ba54ee0a3a3c49a3811c2b54eec37075 b7ed8ec0c3fb469a93d86fe3cfabbcef Stage b7ed8ec0c3fb469a93d86fe3cfabbcef->598343b412db4d5eb5fffdbd7c7f4e7e c662e42262f642baacbd744fb02e58ac Camera c662e42262f642baacbd744fb02e58ac->de852d6bb54e49848fce4d1629a98a74 126aceb3ed6c4e46ae331482b0e35ed2 Log Server 3e96ab3caf434e7d9e011a8047f14b4a Container Registry 850e1ddc3aff4030b36fc3ba1f11c8dc GitHub Actions Runner

Most of the nodes in this diagram are hyperlinks to further documentation of this system.

This system primarily sends messages between services using the STOMP protocul with the Pigeon library. The configuration for this system as a whole is stored on GitHub at AllenInstitute/TEM_config. This repository includes both a Docker compose file for starting all the services that are part of the system, and the configuration files for the services.