Developer

Getting MIES

Latest development version from main branch

• git clone --recurse-submodules https://github.com/AllenInstitute/MIES

• ./tools/initial-repo-config.sh (Requires a Git Bash shell, named Git terminal in SourceTree)

Arduino

Setup

Advanced measurement modes like Yoking require an Arduino for triggering the DAC hardware. The following steps have to be performed in order to get a working setup:

• Get an Arduino UNO, for easier PIN access a screw shield comes in handy too

• Connect the device to the PC via USB

• Install the Arduino studio from “Packages\Arduino\arduino-1.6.8-windows.exe”

• Extract “Packages\Arduino\Arduino-libraries-and-sequencer.zip” into “C:\Users\$username\Documents\Arduino”

• Start Arduino studio and try connecting to the device

• Load and compile the installed sequence “Igor_Sequencer3.ino”

• Connect Pin 12 and GND to the trigger input of the DAC hardware

Usage

• Connect Arduino

• Start Arduino studio and upload “Igor_Sequencer3.ino”

• Start Igor Pro

• Open the panel from the Arduino menu

• Connect

• Upload Sequence

• The start of DAQ is done by MIES itself

Building the documentation

• The script tools/build-documentation.sh allows to build the documentation locally

• This command also installs the required pip packages, so using a dedicated virtual environment is advised

• Linux users might directly execute the docker version in tools/docker/build-documentation.sh

Updating documentation

Due to our excessive use of the breathe sphinx extension which feeds from doxygen, a full documentation build takes around 10 minutes. It is also not possible to use the sphinx autobuild feature, as it rebuilds all everything from scratch due to breathe.

For fast read-write-view cycles while writing the user documentation do the following:

• Start with a clean Packages/doc folder

• Apply the patch which temporarily removes breathe via git am ...

• Call make autobuild which opens a local webbrowser and rebuilds after every change. This time incremental updates work.

Release Handling

If guidelines are not followed, the MIES version will be unknown, and data acquisition is blocked.

Cutting a new release

• Checkout a new branch git checkout -b feature/XXXX-release-notes main

• Paste the contents of Packages\doc\releasenotes_template.rst to the top of Packages\doc\releasenotes.rst

• Call tools\create-changelog.sh which generate a raw changelog and fill releasenotes.rst with a cleaned up version of it.

• Propose a pull request and get it merged

• Checkout the main branch

• Tag the current state with git tag Release_X.Y_*, see git tag for how the asterisk should look like

• Push the tag: git push origin $tag. You can pass --dry-run for testing out what is getting pushed without pushing anything.

• Create the release branches:

  • git checkout -b release/X.Y

  • git push --no-verify -u origin release/X.Y

• Change the bamboo plans using release branches to use the branch release/X.Y

• Create a new release on github and check that the bamboo job correctly uploads the artifacts

Creating a release package manually

• Open a git bash terminal by choosing Actions->”Open in terminal” in SourceTree

• Checkout the release branch git checkout release/$myVersion

• If none exists create one with git checkout -b release/$myVersion

• Change to the tools directory in the worktree root folder

• Execute ./create-release.sh

• The release package including the version information is then available as zip file

Continuous integration server

Our CI server, called bamboo, provides the following services for MIES:

Automatic release package building

• The release branch, release/$number with the highest $number, is polled every 3 minutes for changes

• If changes are detected, a clone is fetched, and inside a checked out git working tree, the release script tools/create-release.sh is executed.

• The result of the release script, called an artifact in CI-speech, is then available as zip package from the Package section.

• The release packaging job can be run on a windows box with git for windows installed. This is ensured by a platform requirement for the job.

Compilation testing

The full MIES installation with and without hardware XOPs are IGOR Pro compile tested using a bamboo job. This allows to catch compile time errors early on.

For testing compilation manually perform the following steps:

• Create in User Procedures a shortcut pointing to Packages\MIES_Include.ipf and Packages\tests

• Remove the shortcut Packages\MIES_Include.ipf in Igor Procedures

• Close all Igor Pro instances

• Execute tools\check_mies_compilation.sh

• Watch the output

Unit and integration testing

A couple of our bamboo jobs are responsible for executing tests. All tests are written using the Igor Unit Testing Framework.

The folders in Packages\tests follow a common naming scheme. Each folder holds a separate Igor Experiment with tests. The tests in folders starting with Hardware requires present hardware, the others don’t. In each folder an Igor Experiment named like the folder with .pxp-suffix is present which allows to execute all the tests from that folder.

For executing the tests manually perform the followings steps:

• Create in User Procedures a shortcut pointing to Packages\MIES_Include.ipf and Packages\tests

• Remove the shortcut Packages\MIES_Include.ipf in Igor Procedures

• Open one of the test experiments in Packages\tests

• Call RunWithOpts()

• Watch the output

The environment variables BAMBOO_INSTRUMENT_TESTS/BAMBOO_EXPENSIVE_CHECKS allow to tweak test execution. By default we do expensive tests in CI and instrumentation in CI for the main branch. Accepted are all numbers but the values 0/1 are suggested.

Documentation building

The documentation for the main branch is automatically built and uploaded by this bamboo job.

Setting up a continous integration server (Linux)

Preliminaries

• Linux box with fixed IP

• Choose a user, here named ci, for running the tests.

• Make sure that the user is not a member of the wheel group so that it can not gain root access.

• Install the bare-minimum packages and use the latest debian stable

Enable SSH access

• Setup remote SSH access with public keys. On the client (your PC!) try logging into using SSH.

• Disable password authentication in /etc/ssh/sshd_config

Install required software

• Install Docker

• Misc required software: apt install git cron-apt xmllint

• Enable automatic updates: echo "dist-upgrade -y -o APT::Get::Show-Upgraded=true" > /etc/cron-apt/action.d/4-upgrade

• Install OpenJDK 8 by adding a file with the following sources in /etc/apt/sources.list.d/:

  deb https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/ buster main
  

• apt update

• apt install adoptopenjdk-8-hotspot-jre

• update-alternatives --config java and select version 8

Setup bamboo agent

• Install the bamboo agent according to the instructions and run it once to create the bamboo-agent-home directory

• Create a file /etc/systemd/system/bamboo.service with the following contents

  [Unit]
  Description=Atlassian Bamboo
  After=syslog.target network.target
  
  [Service]
  Type=forking
  User=ci
  ExecStart=/home/ci/bamboo-agent-home/bin/bamboo-agent.sh start
  ExecStop=/home/ci/bamboo-agent-home/bin/bamboo-agent.sh stop
  SuccessExitStatus=143
  Environment="PATH=/home/ci/.local/bin:/usr/local/bin:/usr/bin:/bin"
  
  [Install]
  WantedBy=multi-user.target
  

• Enable it with systemctl enable bamboo.service

• Reboot the system and check that the agent runs

• Add a fitting Igor Pro (new) capability to the agent in bamboo.

• Make the agent dedicated to the MIES-Igor project.

• Be sure that the “git” capability and the “bash” executable capability are present as well

Setting up a continous integration server (Windows)

• Windows 10 with “Remote Desktop” enabled user

• Install the folllowing programs:

  • Java 8

  • Git (choose the installer option which will make the Unix tools available in cmd as well)

  • Multiclamp Commander

  • NIDAQ-mx driver package 19.0 or later

  • NIDAQ-mx XOP from WaveMetrics

  • HEKA Harware Drivers 2014-03 Windows.zip

  • Igor Pro (latest required versions)

  • Install bamboo remote agent according to http://bamboo.corp.alleninstitute.org/admin/agent/addRemoteAgent.action.

• Start Igor Pro and open a DA_Ephys panel, lock the device. This will not work, so follow the posted suggestions to get it working (registry fix and ASLR fix).

• Add a fitting MIES_AgentType capability to the agent in bamboo.

• Make the agent dedicated to the MIES-Igor project.

• Be sure that the “git” capability and the “bash” executable capability are present as well

• Create the folder $HOME/.credentials and place the file github_api_token from an existing CI machine there

• Copy tools/start-bamboo-agent-windows.sh and tools/start-bamboo-agent-windows.bat to $HOME

• Edit tools/start-bamboo-agent-windows.bat so that it points to the existing Git location

• Add shortcuts to $HOME/start-bamboo-agent-windows.bat and MC700B.exe into C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp

• (Only for Code signing machine): Also add a shortcut to $HOME/start-bamboo-agent-windows.bat in the Startup folder

Available CI servers

Distributing jobs to agents in bamboo is done via our MIES_AgentType capability.

The following capabilities are in use:

• Linux: Two agents run on Linux with

  • Debian 10 (buster)

  • No Hardware

  • No Igor Pro

  • Docker

• CodeSigning: Agent can sign installer packages

  • Windows 10

  • EV certificate on USB stick

• ITC: Agent can execute hardware tests with ITC18USB hardware

  • ITC18-USB hardware, 2 AD/DA channels are looped

  • MCC demo amplifier only

  • Igor Pro (latest required versions)

• NI: Agent can execute hardware tests with NI/ITC1600 hardware

  • ITC-1600 hardware with one rack, 2 AD/DA channels are looped

  • NI PCIe-6343, 2 AD/DA channels are looped

  • MCC demo amplifier only

  • Igor Pro (latest required versions)

Branch naming scheme

For making code review easier we try to follow a naming scheme for branches behind PRs.

Scheme: $prefix/$pr-$text(-backport)?

Where $prefix is one of feature/bugfix, $pr is the number of the soon-to-be-created pull request and $text a user defined descriptive text. -backport must be present for PRs against release branches only.

Contributers are encouraged to install the pre-push git hook from the tools directory. The script tools/nextFreePRNumber.sh can get the soon-to-be-created PR number on the commandline (requires curl and jq) as well.

Continuous Integration Hints

As part of the continuous integration pipeline tests are run. A full test run including the hardware tests tales several hours. Thus, if a lot of pull requests are updated pending test runs could queue up and it might take rather long until results are available.

Thus, for changes where the commits are in a state where no full test run by the CI makes sense it is possible to inhibit the automatic tests. Typically this is the case if the developer commits changes in progress and pushes these for the purpose of a secondary backup or further commit organization. Inhibiting tests for these cases frees testing resources for other pull requests.

To inhibit test runs the key [SKIP CI] has to be added to the commit message.

The key can be removed later easily through a rebase with rewording the commit message. After pushing to the repository the CI queues the tests again for this pull request.

Debugging threadsafe functions

The function DisableThreadsafeSupport() allows to turn off threadsafe support globally. This allows to use the debugger in threadsafe functions. Every MIES features which does not complain via ASSERT() or BUG() is supposed to work without threadsafe support as well.

Preventing Debugger Popup

There exist critical function calls that raise a runtime error. In well-defined circumstances the error condition is evaluated properly afterwards. When debugger is enabled and options are set to “Debug On Error”, then the Debugger will popup on the line where such functions calls take place. This is inconvenient for debugging because the error is intended and properly handled. To prevent the debugger to open the coding convention is:

AssertOnAndClearRTError()
CriticalFunction(); err = getRTError(1)

Notable the second part that clears the RTE must be in the same line and can not be moved to an own function. This coding convention is only valid, if the critical function is expected to raise an runtime error.

Runtime Error / Abort Handling Conventions

Here a coding convention for try / catch / endtry constructs is introduced to prevent common issues like silently clearing unexpected runtime error conditions by using these.

A try / catch / endtry construct catches by specification either

• Runtime errors when AbortOnRTE is encountered between try / catch

• Aborts when encountered between try / catch

The code must take into account the possibility of runtime errors generated by bad code. These unexpected RTEs must not be silently cleared.

For the case, where an RTE is expected from CriticalFunction, the common approach is:

AssertOnAndClearRTError()
try
    CriticalFunction(); AbortOnRTE
catch
    err = ClearRTError()
    ...
endtry

Here pending RTEs are handled before the try. By convention the AbortOnRTE must be placed in the same function as the try / catch / endtry construct. The code between try / catch should only include critical function calls and be kept minimal. The expected RTE condition should be cleared directly after catch.

For the case, where an Abort is expected from CriticalFunction, the common approach is:

try
    CriticalFunction()
catch
    ...
endtry

As Abort does not generate an RTE condition the try / catch / endtry construct leaves any possible unexpected RTE condition pending and no RTE condition is cleared. The programmer might consider evaluating V_AbortCode after catch.

It is recommended to comment in the code before the try what the construct is intended to handle (RTE, Abort or both).

Retrieving Headstage / Channel Information from the LBN

If you would like to retrieve the settings from the last acquisition then look up function like AFH_GetHeadstageFromDAC. It retrieves the correct information under the following conditions:

• Data Acquisition is ongoing or

• Data Acquisition has finished and DAEphys panel was not changed.

This function returns NaN if the active DAC had no associated headstage. The same applies for AFH_GetHeadstageFromADC.

In contrast the functions AFH_GetDACFromHeadstage and AFH_GetADCFromHeadstage return DAC/ADC numbers only for active headstages.

One of the most used functions to retrieve specific information from the LBN is GetLastSettingChannel. The returned wave has NUM_HEADSTAGES + 1 entries. The first NUM_HEADSTAGES entries refer to the headstages whereas the last entry contains all headstage independent data. This is related to the general layout of the LBN, where the headstage is an index of the wave. In the numerical LBN (GetLBNumericalValues) there are columns with DAC/ADC channel information identified by their respective dimension label. For associated DAC <-> ADC channels the number of the DAC and ADC is present in the layers. The first NUM_HEADSTAGES layers refer to the headstages.

Thus, if headstage 3 uses DAC channel 5 and ADC channel 1 for a sweep then in the LBN at index 3 in the DAC column a 3 is present and in the ADC column a 1. Details of the internal data format of the LBN are not required for correct retrieval of that information as MIES provides functions for that:

WAVE/Z numericalValues = BSP_GetLBNWave(graph, LBN_NUMERICAL_VALUES, sweepNumber = sweep)
if(!WaveExists(numericalValues))
   // fitting handling code
endif
[WAVE/Z settings, index] = GetLastSettingChannel(numericalValues, $"", sweep, "Indexing", channelNumber, channelType, entrySourceType)

This call specifies a sweep number, a channel type and a channel number and asks for information from the “Indexing” field. It returns a 1D wave settings and an index, where settings[index] is a Boolean entry telling if indexing was off or on. The value index itself is the headstage number. The index value can also equal NUM_HEADSTAGES when it refers to a headstage independent value.

To find the ADC channel from a DAC channel, the example above can also be setup with channelType = XOP_CHANNEL_TYPE_DAC and LBN entry name “ADC”. This works the same for finding the DAC channel from a ADC channel.

If one just wants the headstage number there is an utility function GetHeadstageForChannel that returns the active headstage for a channel.

The LBN entry Headstage Active is a Boolean entry and marks which headstage was active in a sweep. The Headstage Active can only be set (1) for a headstage that has an associated DAC and ADC channel.

Creating LBN entries for tests

Make/FREE/N=(1, 1, LABNOTEBOOK_LAYER_COUNT) valuesHSA, valuesDAC, valuesADC
Make/T/FREE/N=(1, 1, 1) keys

sweepNo = 0

// HS 0: DAC 2 and ADC 6
// HS 1: DAC 3 and ADC 7
// HS 2+: No DAC/ADC set
valuesDAC[]  = NaN
valuesDAC[0][0][0] = 2 // The layer refers to the headstage number
valuesDAC[0][0][1] = 3
keys[] = "DAC"
ED_AddEntriesToLabnotebook(valuesDAC, keys, sweepNo, device, DATA_ACQUISITION_MODE)

valuesADC[]  = NaN
valuesADC[0][0][0] = 6
valuesADC[0][0][1] = 7
keys[] = "ADC"
ED_AddEntriesToLabnotebook(valuesADC, keys, sweepNo, device, DATA_ACQUISITION_MODE)

valuesHSA[]  = 0
valuesHSA[0][0][0] = 1 // the only valid option here is to set HS 0 and 1 active
valuesHSA[0][0][1] = 1 // because we did not set ADC/DAC channels for the other HS.
keys[] = "Headstage Active"
ED_AddEntriesToLabnotebook(valuesHSA, keys, sweepNo, device, DATA_ACQUISITION_MODE)

The key function here is ED_AddEntriesToLabnotebook. There are no checks applied for this way of creating LBN entries for tests that guarantee a consistent LBN. e.g. setting headstage 2 to active in the upper code would violate LBN format schema. Note that in contrast ED_AddEntryToLabnotebook is used to add specific user entries to the LBN and is not suited for setting up generic test LBN entries. More example code can be found in PrepareLBN_IGNORE in UTF_Labnotebook.ipf.

Original Developer Docu on SweepFormula

Notebook scans from Matthias (ukos-git):

The following information is taken from annotated excerpts from the above:

• getter functions

• Return data array with sweep and channel information

• Data slicing with 2d location functions

• no comment given

• Plot

• Functions reducing a dimension of data

• Note on time aka xvalue concatenation logics

Adding support for new NI hardware

Newly added NI hardware must fulfill the following properties:

• Allow 500kHz sampling rate for one AI/AO channel

• At least one port of each type: AI/AO/DIO

• Supported by the NIDAQmx XOP and our use of it

To add new hardware:

• Visit the NI website and check if the device fullfills our minimum requirements

• Ask the user to send you the output of HW_NI_PrintPropertiesOfDevices()

• Add that info to NI_DAC_PATTERNS

• Update Readme.md