TEM Architecture
TEM Architecture

pytemca

AllenInstitute/pytemca

%3 cluster_pyTEM States pyTEM States 28749dcefa4e4cffa4f2eedf328e208f Preview/Setup State a05790ae557449f894012e3ff8667d4f ROI Initialization State 28749dcefa4e4cffa4f2eedf328e208f->a05790ae557449f894012e3ff8667d4f 07e8a185f55d4214a1d48d6bf315aa44 Acquisition State a05790ae557449f894012e3ff8667d4f->07e8a185f55d4214a1d48d6bf315aa44 4675f57e3fa14d758dde480172df1a4e Error State a05790ae557449f894012e3ff8667d4f->4675f57e3fa14d758dde480172df1a4e Error 07e8a185f55d4214a1d48d6bf315aa44->28749dcefa4e4cffa4f2eedf328e208f Abort 07e8a185f55d4214a1d48d6bf315aa44->a05790ae557449f894012e3ff8667d4f Repeat 07e8a185f55d4214a1d48d6bf315aa44->4675f57e3fa14d758dde480172df1a4e Error

Preview State

This state is is the entry point to the system. It is the default state at start up and will also be the state that the system will revert to upon detecting an error that does not crash the system.

Preview State Logic

Upon entering this state the system will continuously acquire images from the XIMEA camera and each image frame will be minimally processed. The images are then JPEG encoded and sent to python which pushes them to a websocket to display on a browser based UI.

%3 cluster_opencv_graph opencv_graph cluster_pyTEMCA pyTEMCA fbb8d95aa04949d38d320e6b3a257d3f Acquire Image 63d138429fd842b88997ef0a5e52a81d Flat Field Correction and CLAHE (CUDA) fbb8d95aa04949d38d320e6b3a257d3f->63d138429fd842b88997ef0a5e52a81d 600fc3bfbecc40c285debaae5ee528d7 Calculate Stats (Min/Max/Mean) 63d138429fd842b88997ef0a5e52a81d->600fc3bfbecc40c285debaae5ee528d7 9ed65fae89494d949af51ab1a8d7b2fc Calculate FFT Focus Score 63d138429fd842b88997ef0a5e52a81d->9ed65fae89494d949af51ab1a8d7b2fc bcc96a8f2ec84c85bdd3c1192ae646e6 JPEG Compress 63d138429fd842b88997ef0a5e52a81d->bcc96a8f2ec84c85bdd3c1192ae646e6 63b50cd202e449a1969a7c31635ca711 Send to Python via pybind11 600fc3bfbecc40c285debaae5ee528d7->63b50cd202e449a1969a7c31635ca711 9ed65fae89494d949af51ab1a8d7b2fc->63b50cd202e449a1969a7c31635ca711 bcc96a8f2ec84c85bdd3c1192ae646e6->63b50cd202e449a1969a7c31635ca711 46326ef27b9c468f864036a836a756f3 Encode Image 63b50cd202e449a1969a7c31635ca711->46326ef27b9c468f864036a836a756f3 1e77471447234a2ab5d252dc4cbe8fb6 Publish Image 46326ef27b9c468f864036a836a756f3->1e77471447234a2ab5d252dc4cbe8fb6 c839bef29ad747e9b63973cbe7a7327e Web UI 1e77471447234a2ab5d252dc4cbe8fb6->c839bef29ad747e9b63973cbe7a7327e d62013baeb5b4ebcab43e7140bab87f1 State Changed? d62013baeb5b4ebcab43e7140bab87f1->fbb8d95aa04949d38d320e6b3a257d3f d62013baeb5b4ebcab43e7140bab87f1->d62013baeb5b4ebcab43e7140bab87f1 Repeat

ROI Initialization State

The ROI Initialization State represents a state for moving to an aperture on a grid tape or stick16.

ROI Initialization State Logic

Upon entering this state it handles the entry logic for the state and initializes tape_ranges if it is None. It tries to fetch barcode values from TEM_db (if using tape); otherwise, it sets default values.

Next the code performs the following:

  1. Initialization and Logging: Clears lists related to aperture critical and error events. Logs the action of changing the aperture.

  2. State Check for Pause: Checks if the current montage state is PAUSE. If yes, it schedules the _change_aperture method to run again after 0.2 seconds and exits the current execution.

  3. Preparatory Actions: Updates beam center results and informs the system that a new montage is starting.

  4. Abort Conditions Check: Checks several conditions to decide whether to abort the operation:
    • If required media information is missing.
    • If environmental conditions are not suitable.
    • If there’s a pending request to abort at the end of the montage.
    • If there is insufficient disk space.
    • If too many errors have accumulated (default is 3).
  5. Handling of Tape and Aperture: Retrieves the next task object (TAO) from the montage controller. If a valid TAO is found:
    • Checks if the destination directory exists or needs to be created.
    • Checks for existing data that might require an abort.
    • Performs adjustments based on the first Region of Interest (ROI) from tao.

  6. Aperture Movement and Correction: Manages movement to the correct grid or aperture based on the tape controller status. Deals with potential barcode errors and adjusts based on confidence levels in the barcode readings.

  7. Lens and Beam Adjustments: Adjusts the beam’s brightness and attempts to find a centroid for the aperture if needed. Conducts a quality check on the beam’s brightness and might abort if the beam mean value is too low. Manages auto-centering of the beam if necessary.

  8. Brightfield Imaging: Handles brightfield imaging conditions and might abort based on the quality of consecutive brightfield images.

  9. Lens Correction Montage: Decides whether a new lens correction montage is necessary based on time intervals and changes in lens conditions.

  10. Finalizing and Error Handling: If no ROIs are found in tao, logs the event but continues processing. If there are still tasks (taos) left in the montage queue, it triggers the start of another attempt. Sets the montage state to NONE if no tasks are left.
%3 cluster_ROI Initialization State ROI Initialization State 76a8ba88fd6349fd93920515665db13b Enter State 37bac50f9df84945a89d94d2d48494a1 Initialize tape_ranges 76a8ba88fd6349fd93920515665db13b->37bac50f9df84945a89d94d2d48494a1 56ea64b2cda742c69ec7ca41e1ebebf7 Fetch Barcode from TEM_db 37bac50f9df84945a89d94d2d48494a1->56ea64b2cda742c69ec7ca41e1ebebf7 4cfe5297afa5443f96c69b432749ca1a Initialization & Logging 37bac50f9df84945a89d94d2d48494a1->4cfe5297afa5443f96c69b432749ca1a e745b4ef499e4e5aa5ad56375f4ed7a7 Check if State is PAUSE 4cfe5297afa5443f96c69b432749ca1a->e745b4ef499e4e5aa5ad56375f4ed7a7 4b6f6558336042f9944392414617028a Retry after 0.2s e745b4ef499e4e5aa5ad56375f4ed7a7->4b6f6558336042f9944392414617028a c6de912ff3154c61a26f522400a81ef6 Preparatory Actions e745b4ef499e4e5aa5ad56375f4ed7a7->c6de912ff3154c61a26f522400a81ef6 4b6f6558336042f9944392414617028a->e745b4ef499e4e5aa5ad56375f4ed7a7 93d90ecbb877400891e7a3138ef1f35f Check Abort Conditions c6de912ff3154c61a26f522400a81ef6->93d90ecbb877400891e7a3138ef1f35f be284572960d4ecc8cc315d27d54c95d Handle Tape & Aperture 93d90ecbb877400891e7a3138ef1f35f->be284572960d4ecc8cc315d27d54c95d 3970c985a5164013b9f56ac39ecdba9e Aperture Movement & Correction be284572960d4ecc8cc315d27d54c95d->3970c985a5164013b9f56ac39ecdba9e 281fef2b14c14f72a4523ac3580d25ef Lens & Beam Adjustments 3970c985a5164013b9f56ac39ecdba9e->281fef2b14c14f72a4523ac3580d25ef 9192526418b846038c2f9e3545cec466 Brightfield Imaging 281fef2b14c14f72a4523ac3580d25ef->9192526418b846038c2f9e3545cec466 8d3fb9ef74a94b128295e4cb8116e50c Lens Correction Montage 9192526418b846038c2f9e3545cec466->8d3fb9ef74a94b128295e4cb8116e50c 84f5ae81fc9c41a196a655aa3e5dd586 Finalizing & Error Handling 8d3fb9ef74a94b128295e4cb8116e50c->84f5ae81fc9c41a196a655aa3e5dd586 cf11c22902dd404485d699d57c390b88 Set State to NONE 84f5ae81fc9c41a196a655aa3e5dd586->cf11c22902dd404485d699d57c390b88

Acquisition State

The Acquisition State represents the overall state of the acquisition loop in the system for imaging and processing data. It manages tasks related to moving the stage, capturing images, processing metadata, and handling various states during the acquisition process.

Acquisition State Logic

Main Acquisition Loop Activation

The acquisition starts if both the main_acquisition_loop_active_event is set and _start_new_aperture() returns True. Initialize variables and prepare for the acquisition sequence within a specific aperture.

ROI Acquisition Process

Iterate over each ROI in roi_list.

For each ROI:

  • Initialize the ROI and calculate the starting position.
  • Move the stage to the initial position and wait for it to stabilize.
  • Enter a nested loop to handle image capture for each tile in the ROI until there are no more positions to move to.
    • Set metadata for the image.
    • Start camera exposure.
    • If there was a previous image, wait for asynchronous processes to complete and possibly write metadata.
    • Wait for the current exposure to complete.
    • Get the next position and move the stage.
    • Handle synchronization and stabilization of the stage.
  • After completing all tiles in the ROI, perform final processing such as writing metadata and resetting the stage to the starting position.

Lens Correction Process

If lens correction is part of the current montage, attempt to correct lens distortions using template matching and solving algorithms. Multiple attempts are made, and if unsuccessful, a critical error event is logged.

Completion and Metadata Handling

Finalize the metadata file and write results to a database. Create a robocopy file if not aborting.

Abort or Switch Event

If an abort signal is received or it’s time to switch to another aperture, the loop is terminated or adjusted accordingly.

Event Scheduling

Schedule callbacks for events like “BACK_TO_SETUP” or “NEXT_APERTURE” based on the conditions met during the loop.

%3 cluster_Outer Infinite Loop Outer Infinite Loop cluster_Main Acquisition Main Acquisition cluster_ROI Acquisition Process ROI Acquisition Process cluster_Per-Tile Per-Tile cluster_Lens Correction Process Lens Correction Process cluster_Completion & Metadata Handling Completion & Metadata Handling cluster_Abort or Switch Event Abort or Switch Event cluster_Event Scheduling Event Scheduling c179185734a2484e8a83721e0a2b90a2 Start Outer Loop 4ac574fc4b814e3c8e3ffeb72c9c7965 Check Activation & Start c179185734a2484e8a83721e0a2b90a2->4ac574fc4b814e3c8e3ffeb72c9c7965 0b4677127f3a48c68640ad3585c08088 End Outer Loop 0b4677127f3a48c68640ad3585c08088->c179185734a2484e8a83721e0a2b90a2 1ca3524ca02a44ee8c29296a0f71b5c5 Initialize & Prepare Acquisition 4ac574fc4b814e3c8e3ffeb72c9c7965->1ca3524ca02a44ee8c29296a0f71b5c5 e5d25ff914e948a3b1189798f6040bd5 For Each ROI 1ca3524ca02a44ee8c29296a0f71b5c5->e5d25ff914e948a3b1189798f6040bd5 5ff83110a67d4f20a943397ff22bf955 Initialize ROI & Calculate Position e5d25ff914e948a3b1189798f6040bd5->5ff83110a67d4f20a943397ff22bf955 cd10fa07d7cf4e0cb17ac72f5adff59a Move & Stabilize Stage 5ff83110a67d4f20a943397ff22bf955->cd10fa07d7cf4e0cb17ac72f5adff59a 34eef3eacf0c48cea482d2ce5f3a532a Set Metadata for Each Tile cd10fa07d7cf4e0cb17ac72f5adff59a->34eef3eacf0c48cea482d2ce5f3a532a 2ac8b1dae98e43dcba7121da49c80344 Finalize ROI 951824a815e34605a1344a94624fdfa4 Correct Lens Distortions 2ac8b1dae98e43dcba7121da49c80344->951824a815e34605a1344a94624fdfa4 fa06da404904498cae94685844947822 Start Camera Exposure 34eef3eacf0c48cea482d2ce5f3a532a->fa06da404904498cae94685844947822 87f9109aaebc4f468725e16445ff839f Handle Async & Write Metadata fa06da404904498cae94685844947822->87f9109aaebc4f468725e16445ff839f d319d6a8b34c4d45a5006d275e0ffad9 Complete Exposure 87f9109aaebc4f468725e16445ff839f->d319d6a8b34c4d45a5006d275e0ffad9 2f66db88ccc24a66975aa0e11241b430 Get Next Position d319d6a8b34c4d45a5006d275e0ffad9->2f66db88ccc24a66975aa0e11241b430 93bb40bf05f84d3ebcf6005eac42b61a Stabilize Stage 2f66db88ccc24a66975aa0e11241b430->93bb40bf05f84d3ebcf6005eac42b61a a4e7b8f6976c437c9289a402ae16da6a More Tiles? 93bb40bf05f84d3ebcf6005eac42b61a->a4e7b8f6976c437c9289a402ae16da6a a4e7b8f6976c437c9289a402ae16da6a->2ac8b1dae98e43dcba7121da49c80344 a4e7b8f6976c437c9289a402ae16da6a->34eef3eacf0c48cea482d2ce5f3a532a c7d38a6898924df095b91082facbdf7c Finalize Metadata 951824a815e34605a1344a94624fdfa4->c7d38a6898924df095b91082facbdf7c a5860971c896452ab3253beb9d689694 Create Robocopy File c7d38a6898924df095b91082facbdf7c->a5860971c896452ab3253beb9d689694 d95dc3e782964cf4bc97abd08d84859c Check for Abort or Switch a5860971c896452ab3253beb9d689694->d95dc3e782964cf4bc97abd08d84859c 5f285a4b47e94f91b6ec7d0846ca103c Handle Abort d95dc3e782964cf4bc97abd08d84859c->5f285a4b47e94f91b6ec7d0846ca103c 01e6d135b3ac490f8dc8b073a2ad1862 Schedule Events 5f285a4b47e94f91b6ec7d0846ca103c->01e6d135b3ac490f8dc8b073a2ad1862 01e6d135b3ac490f8dc8b073a2ad1862->0b4677127f3a48c68640ad3585c08088

Error State