TEM Architecture
TEM Architecture

pytemca

AllenInstitute/pytemca

%3 cluster_pyTEM States pyTEM States 7107ef3bffad48f892edbbddb20664e8 Preview/Setup State 245bae3ec0764d30a5081067f3e2c4a3 ROI Initialization State 7107ef3bffad48f892edbbddb20664e8->245bae3ec0764d30a5081067f3e2c4a3 1cf90a15b91a4fdf98295d2d3ded274d Acquisition State 245bae3ec0764d30a5081067f3e2c4a3->1cf90a15b91a4fdf98295d2d3ded274d a2592c65a436416fbf3ea37edf3f0355 Error State 245bae3ec0764d30a5081067f3e2c4a3->a2592c65a436416fbf3ea37edf3f0355 Error 1cf90a15b91a4fdf98295d2d3ded274d->7107ef3bffad48f892edbbddb20664e8 Abort 1cf90a15b91a4fdf98295d2d3ded274d->245bae3ec0764d30a5081067f3e2c4a3 Repeat 1cf90a15b91a4fdf98295d2d3ded274d->a2592c65a436416fbf3ea37edf3f0355 Error

Preview State

This state is is the entry point to the system. It is the default state at start up and will also be the state that the system will revert to upon detecting an error that does not crash the system.

Preview State Logic

Upon entering this state the system will continuously acquire images from the XIMEA camera and each image frame will be minimally processed. The images are then JPEG encoded and sent to python which pushes them to a websocket to display on a browser based UI.

%3 cluster_opencv_graph opencv_graph cluster_pyTEMCA pyTEMCA 1d89710ed3bf4cd9aea41e5b70b57c8a Acquire Image 69e59e49d3ff415dbe719446e292c442 Flat Field Correction and CLAHE (CUDA) 1d89710ed3bf4cd9aea41e5b70b57c8a->69e59e49d3ff415dbe719446e292c442 3bd175af019e4e279d74bb38183bef25 Calculate Stats (Min/Max/Mean) 69e59e49d3ff415dbe719446e292c442->3bd175af019e4e279d74bb38183bef25 565e572337e340c2bbe25e70e89a90e5 Calculate FFT Focus Score 69e59e49d3ff415dbe719446e292c442->565e572337e340c2bbe25e70e89a90e5 602e4846972e4537a97f76a920cada49 JPEG Compress 69e59e49d3ff415dbe719446e292c442->602e4846972e4537a97f76a920cada49 df7749f1988845f991cde46946d06683 Send to Python via pybind11 3bd175af019e4e279d74bb38183bef25->df7749f1988845f991cde46946d06683 565e572337e340c2bbe25e70e89a90e5->df7749f1988845f991cde46946d06683 602e4846972e4537a97f76a920cada49->df7749f1988845f991cde46946d06683 b7e4765cdd5e46da8c01a8d129bd3eb5 Encode Image df7749f1988845f991cde46946d06683->b7e4765cdd5e46da8c01a8d129bd3eb5 7628f2a2f2544c2dac61e83b1e63b0ba Publish Image b7e4765cdd5e46da8c01a8d129bd3eb5->7628f2a2f2544c2dac61e83b1e63b0ba 14bb457c14d64d34a74993e5998c11e5 Web UI 7628f2a2f2544c2dac61e83b1e63b0ba->14bb457c14d64d34a74993e5998c11e5 79a796c6819649fbae973d391e9ac91e State Changed? 79a796c6819649fbae973d391e9ac91e->1d89710ed3bf4cd9aea41e5b70b57c8a 79a796c6819649fbae973d391e9ac91e->79a796c6819649fbae973d391e9ac91e Repeat

ROI Initialization State

The ROI Initialization State represents a state for moving to an aperture on a grid tape or stick16.

ROI Initialization State Logic

Upon entering this state it handles the entry logic for the state and initializes tape_ranges if it is None. It tries to fetch barcode values from TEM_db (if using tape); otherwise, it sets default values.

Next the code performs the following:

  1. Initialization and Logging: Clears lists related to aperture critical and error events. Logs the action of changing the aperture.

  2. State Check for Pause: Checks if the current montage state is PAUSE. If yes, it schedules the _change_aperture method to run again after 0.2 seconds and exits the current execution.

  3. Preparatory Actions: Updates beam center results and informs the system that a new montage is starting.

  4. Abort Conditions Check: Checks several conditions to decide whether to abort the operation:
    • If required media information is missing.
    • If environmental conditions are not suitable.
    • If there’s a pending request to abort at the end of the montage.
    • If there is insufficient disk space.
    • If too many errors have accumulated (default is 3).
  5. Handling of Tape and Aperture: Retrieves the next task object (TAO) from the montage controller. If a valid TAO is found:
    • Checks if the destination directory exists or needs to be created.
    • Checks for existing data that might require an abort.
    • Performs adjustments based on the first Region of Interest (ROI) from tao.

  6. Aperture Movement and Correction: Manages movement to the correct grid or aperture based on the tape controller status. Deals with potential barcode errors and adjusts based on confidence levels in the barcode readings.

  7. Lens and Beam Adjustments: Adjusts the beam’s brightness and attempts to find a centroid for the aperture if needed. Conducts a quality check on the beam’s brightness and might abort if the beam mean value is too low. Manages auto-centering of the beam if necessary.

  8. Brightfield Imaging: Handles brightfield imaging conditions and might abort based on the quality of consecutive brightfield images.

  9. Lens Correction Montage: Decides whether a new lens correction montage is necessary based on time intervals and changes in lens conditions.

  10. Finalizing and Error Handling: If no ROIs are found in tao, logs the event but continues processing. If there are still tasks (taos) left in the montage queue, it triggers the start of another attempt. Sets the montage state to NONE if no tasks are left.
%3 cluster_ROI Initialization State ROI Initialization State a1d3fad803d54670990910e2b649088f Enter State 852d0be4ad06401b974b67d40e83332a Initialize tape_ranges a1d3fad803d54670990910e2b649088f->852d0be4ad06401b974b67d40e83332a 8c95bc207cc544e2a3c73b39d4e21668 Fetch Barcode from TEM_db 852d0be4ad06401b974b67d40e83332a->8c95bc207cc544e2a3c73b39d4e21668 1a3f2f3ed076444ea400289739fe4920 Initialization & Logging 852d0be4ad06401b974b67d40e83332a->1a3f2f3ed076444ea400289739fe4920 94e32ac21e024716bd36d1989ea28fcb Check if State is PAUSE 1a3f2f3ed076444ea400289739fe4920->94e32ac21e024716bd36d1989ea28fcb 3a79bccf2ecf42668cdff4f0c538958a Retry after 0.2s 94e32ac21e024716bd36d1989ea28fcb->3a79bccf2ecf42668cdff4f0c538958a cc760224a5af4df8bdf033e4a48709b3 Preparatory Actions 94e32ac21e024716bd36d1989ea28fcb->cc760224a5af4df8bdf033e4a48709b3 3a79bccf2ecf42668cdff4f0c538958a->94e32ac21e024716bd36d1989ea28fcb 190f14945b654544a4f1bc71fcdf4785 Check Abort Conditions cc760224a5af4df8bdf033e4a48709b3->190f14945b654544a4f1bc71fcdf4785 71a007e327f44f8ab2b11c1bf1750b47 Handle Tape & Aperture 190f14945b654544a4f1bc71fcdf4785->71a007e327f44f8ab2b11c1bf1750b47 e5ba9edbe9674705aeb9742e5c87d956 Aperture Movement & Correction 71a007e327f44f8ab2b11c1bf1750b47->e5ba9edbe9674705aeb9742e5c87d956 4bffdf9175f24853a13556dcb448f06d Lens & Beam Adjustments e5ba9edbe9674705aeb9742e5c87d956->4bffdf9175f24853a13556dcb448f06d 822f65f4614f44c2a5ea65306d146383 Brightfield Imaging 4bffdf9175f24853a13556dcb448f06d->822f65f4614f44c2a5ea65306d146383 4cd52d761d034d5cba4ff400572a1e4a Lens Correction Montage 822f65f4614f44c2a5ea65306d146383->4cd52d761d034d5cba4ff400572a1e4a ace6db4768634220a685cccaa2c2a7c4 Finalizing & Error Handling 4cd52d761d034d5cba4ff400572a1e4a->ace6db4768634220a685cccaa2c2a7c4 31bcee3b3e104d50ada14216229c66da Set State to NONE ace6db4768634220a685cccaa2c2a7c4->31bcee3b3e104d50ada14216229c66da

Acquisition State

The Acquisition State represents the overall state of the acquisition loop in the system for imaging and processing data. It manages tasks related to moving the stage, capturing images, processing metadata, and handling various states during the acquisition process.

Acquisition State Logic

Main Acquisition Loop Activation

The acquisition starts if both the main_acquisition_loop_active_event is set and _start_new_aperture() returns True. Initialize variables and prepare for the acquisition sequence within a specific aperture.

ROI Acquisition Process

Iterate over each ROI in roi_list.

For each ROI:

  • Initialize the ROI and calculate the starting position.
  • Move the stage to the initial position and wait for it to stabilize.
  • Enter a nested loop to handle image capture for each tile in the ROI until there are no more positions to move to.
    • Set metadata for the image.
    • Start camera exposure.
    • If there was a previous image, wait for asynchronous processes to complete and possibly write metadata.
    • Wait for the current exposure to complete.
    • Get the next position and move the stage.
    • Handle synchronization and stabilization of the stage.
  • After completing all tiles in the ROI, perform final processing such as writing metadata and resetting the stage to the starting position.

Lens Correction Process

If lens correction is part of the current montage, attempt to correct lens distortions using template matching and solving algorithms. Multiple attempts are made, and if unsuccessful, a critical error event is logged.

Completion and Metadata Handling

Finalize the metadata file and write results to a database. Create a robocopy file if not aborting.

Abort or Switch Event

If an abort signal is received or it’s time to switch to another aperture, the loop is terminated or adjusted accordingly.

Event Scheduling

Schedule callbacks for events like “BACK_TO_SETUP” or “NEXT_APERTURE” based on the conditions met during the loop.

%3 cluster_Outer Infinite Loop Outer Infinite Loop cluster_Main Acquisition Main Acquisition cluster_ROI Acquisition Process ROI Acquisition Process cluster_Per-Tile Per-Tile cluster_Lens Correction Process Lens Correction Process cluster_Completion & Metadata Handling Completion & Metadata Handling cluster_Abort or Switch Event Abort or Switch Event cluster_Event Scheduling Event Scheduling 90713aef95b3472ea718cab67d7329af Start Outer Loop f3d29fdadd6345dbae572833f947c863 Check Activation & Start 90713aef95b3472ea718cab67d7329af->f3d29fdadd6345dbae572833f947c863 e5148dfb2134453090da1638c991b221 End Outer Loop e5148dfb2134453090da1638c991b221->90713aef95b3472ea718cab67d7329af 1863f8a34bec4ae8889adda7bfe9a069 Initialize & Prepare Acquisition f3d29fdadd6345dbae572833f947c863->1863f8a34bec4ae8889adda7bfe9a069 47c895d2eb604e5ca5085ad5220c99bd For Each ROI 1863f8a34bec4ae8889adda7bfe9a069->47c895d2eb604e5ca5085ad5220c99bd 5ecd290b45d8432ba27f80bc009f062b Initialize ROI & Calculate Position 47c895d2eb604e5ca5085ad5220c99bd->5ecd290b45d8432ba27f80bc009f062b eb67535d8278440bb37ee8c8c4bbf07f Move & Stabilize Stage 5ecd290b45d8432ba27f80bc009f062b->eb67535d8278440bb37ee8c8c4bbf07f fb7f0a65175a4175af410c7b7b92b095 Set Metadata for Each Tile eb67535d8278440bb37ee8c8c4bbf07f->fb7f0a65175a4175af410c7b7b92b095 431005f59f0a43e782e75fcd177ff5af Finalize ROI 13152b11ac32416092bc233675192a57 Correct Lens Distortions 431005f59f0a43e782e75fcd177ff5af->13152b11ac32416092bc233675192a57 0bd60a20b864455f9b1410849e2d837d Start Camera Exposure fb7f0a65175a4175af410c7b7b92b095->0bd60a20b864455f9b1410849e2d837d fda110c4a17b4ed39a240b16853737a6 Handle Async & Write Metadata 0bd60a20b864455f9b1410849e2d837d->fda110c4a17b4ed39a240b16853737a6 ec5061b727ec4a2c985397aebd709414 Complete Exposure fda110c4a17b4ed39a240b16853737a6->ec5061b727ec4a2c985397aebd709414 e244789381254236b195e92f2e8e38a2 Get Next Position ec5061b727ec4a2c985397aebd709414->e244789381254236b195e92f2e8e38a2 df7e2a3ad7a8464fb1092795b11a452c Stabilize Stage e244789381254236b195e92f2e8e38a2->df7e2a3ad7a8464fb1092795b11a452c 3eb7e0b122374674a17ff51dc596f11b More Tiles? df7e2a3ad7a8464fb1092795b11a452c->3eb7e0b122374674a17ff51dc596f11b 3eb7e0b122374674a17ff51dc596f11b->431005f59f0a43e782e75fcd177ff5af 3eb7e0b122374674a17ff51dc596f11b->fb7f0a65175a4175af410c7b7b92b095 6250f54e62f34c59811609c1f446a897 Finalize Metadata 13152b11ac32416092bc233675192a57->6250f54e62f34c59811609c1f446a897 38383ed345bb43ea98a107e48e94ff81 Create Robocopy File 6250f54e62f34c59811609c1f446a897->38383ed345bb43ea98a107e48e94ff81 93cbed6d3a70499fab8c3cf2ba9b4dd2 Check for Abort or Switch 38383ed345bb43ea98a107e48e94ff81->93cbed6d3a70499fab8c3cf2ba9b4dd2 7ea6d142150d4ef9bb0bebef72ad2298 Handle Abort 93cbed6d3a70499fab8c3cf2ba9b4dd2->7ea6d142150d4ef9bb0bebef72ad2298 89e2cc0379ff4c5ea9c5c100393a04e0 Schedule Events 7ea6d142150d4ef9bb0bebef72ad2298->89e2cc0379ff4c5ea9c5c100393a04e0 89e2cc0379ff4c5ea9c5c100393a04e0->e5148dfb2134453090da1638c991b221

Error State