Rebuilding and Deploying the Developing Mouse Visual Cortex Gene Expression Viewer

This guide documents the complete, reproducible workflow used to convert the public Developing Mouse Visual Cortex files from ABC Atlas Access into the pseudobulk and gene-level files required by the web application, publish those files to public AWS storage, run the R Shiny application, deploy it to Posit Connect, and configure GitHub to update the deployed application automatically.

Scope: This documentation applies only to the mouse-developing-vis-app repository and Developing Mouse Visual Cortex viewer. It does not document the SEA-AD Multiregion application.

Source repository: AllenInstitute/mouse-developing-vis-app
Live application: Developing Mouse Visual Cortex Gene Expression Viewer
Data documentation: ABC Atlas Access: Developing Mouse Visual Cortex

End-to-end workflow

  1. Create a GitHub repository and clone it into a linked Code Ocean Capsule.
  2. Configure a Python + R Code Ocean environment with all compiled and interpreted dependencies.
  3. Run create_dev_mouse_vis_cortex_pseudobulk.py to produce a pseudobulk H5AD.
  4. Capture the H5AD as a Code Ocean Data Asset and mount it under /data.
  5. Run create_mouse_aging_parquet.py to create gene-level Parquet and supporting metadata, excluding the statistics table.
  6. Run create_statistics.py to create the final exploratory gene-statistics table.
  7. Run create_value_sets.py to create authoritative category order and color definitions.
  8. Upload the app-ready directory from /results to a public S3 prefix and verify parity.
  9. Test code/shiny/app.R against the public S3 files.
  10. Create and validate a reproducible renv.lock.
  11. Deploy the app to Posit Connect.
  12. Configure GitHub Actions to regenerate the deployment manifest from renv.lock and update the existing Connect content after changes under code/shiny/.
  13. Publish index.html and this deployment guide with GitHub Pages.

Repository structure

mouse-developing-vis-app/
├── .github/
│   └── workflows/
│       └── deploy-connect.yml
├── code/
│   ├── create_dev_mouse_vis_cortex_pseudobulk.py
│   ├── create_mouse_aging_parquet.py
│   ├── create_statistics.py
│   ├── create_value_sets.py
│   ├── run
│   └── shiny/
│       ├── app.R
│       ├── abc_atlas_gene_url_function.R
│       ├── abc_atlas_schema_kiwi.txt
│       ├── table_column_definitions.csv
│       ├── renv.lock
│       ├── .Rprofile
│       ├── .rscignore
│       └── www/
│           ├── Developing_Mouse_Visual_Cortex_logo.png
│           ├── allen_institute_logo.svg
│           └── tokens.csv
├── data/
│   └── .gitkeep
├── index.html
├── deployment_documentation.html
├── README.md
├── requirements.txt
└── .gitignore

Downloaded expression matrices, caches, H5AD files, Parquet files, and intermediate results are not committed to GitHub.


1Set up the Code Ocean Capsule

Create the GitHub repository

  1. Create a repository named mouse-developing-vis-app in the appropriate GitHub organization.
  2. Add the repository structure shown above.
  3. Keep data/.gitkeep empty so Git tracks the otherwise empty directory.
  4. Use the Python .gitignore template and add exclusions for large scientific files.
# ABC Atlas cache and generated scientific data
data/*
!data/.gitkeep
results/
abc_atlas_cache/
*.h5ad
*.h5
*.parquet

# R project state
.Rhistory
.RData
.Ruserdata
.Rproj.user/
renv/library/
renv/staging/

# Python state
__pycache__/
*.py[cod]
.ipynb_checkpoints/
.venv/
venv/

Create .gitattributes to keep runtime scripts compatible with Linux:

code/run text eol=lf
*.sh text eol=lf
*.py text eol=lf

Clone the repository into Code Ocean

  1. Add GitHub credentials to the Code Ocean account. For a private organizational repository, ensure that the GitHub credential is authorized for the organization.
  2. In Code Ocean, choose New Capsule → Clone from Git.
  3. Enter the HTTPS GitHub repository URL.
  4. Name the Capsule Developing Mouse Visual Cortex Gene Expression Viewer.
  5. Use Clone from Git, not Copy from Git, so the Capsule retains its repository connection.

Code Ocean maps the repository's code/ and data/ directories into the Capsule's corresponding sections. After cloning, open a terminal and verify the remote and branch:

cd /root/capsule
git remote -v
git branch --show-current
git status

If the repository is renamed, update the remote explicitly rather than relying on a redirect:

git remote set-url origin https://github.com/AllenInstitute/mouse-developing-vis-app.git
git fetch origin
git push --dry-run origin main

Configure the Python + R environment

Select a current Python + R starter environment. The validated workflow used Python 3.12 and R 4.3.3. Add dependencies through the Environment Editor, rebuild, commit the environment, and launch a fresh workstation after changes.

Python packages

git+https://github.com/AllenInstitute/abc_atlas_access.git
anndata
h5py
numpy
pandas
scipy
pyarrow

R CRAN packages

shiny
bslib
dplyr
ggplot2==3.5.2
scales
shinycssloaders
cachem
DT
jsonlite
renv
rsconnect
curl

Conda/Mamba packages

r-arrow
libcurl
openssl
libstdcxx-ng>=13
libgcc-ng>=13
nodejs
awscli

Node package and environment variable

The gene-specific ABC Atlas link uses the Node package kiwi-schema. This is an npm package, not a CRAN, Bioconductor, or pip package. In the Code Ocean Environment Editor, add the following post-install command:

npm install --global kiwi-schema@0.5.0

Add this environment variable so Node can resolve globally installed modules:

NODE_PATH=/opt/conda/lib/node_modules

After rebuilding the environment, confirm all three commands succeed:

node --version
npm --version
node -e "console.log(require.resolve('kiwi-schema'))"
Compiled-library consistency: Install R Arrow and its C++ runtime through Conda/Mamba. Use the compatible ggplot2 3.5.2 stack for R 4.3.3. Install the R curl package against the current Conda libcurl and OpenSSL environment. Do not create compatibility symlinks between incompatible OpenSSL or C++ runtime versions.

Environment variable

LD_LIBRARY_PATH=/opt/conda/lib:/usr/local/lib:/usr/lib/x86_64-linux-gnu:/lib/x86_64-linux-gnu

After rebuilding, validate the environment before running project code:

python - <<'PY'
import anndata, h5py, numpy, pandas, scipy, pyarrow
from abc_atlas_access.abc_atlas_cache.abc_project_cache import AbcProjectCache
print("Python imports succeeded")
PY

Rscript -e 'library(arrow); library(ggplot2); library(curl); print(arrow::arrow_info()); print(curl::curl_version())'

Use /data and /results correctly

PathPurposePersistence pattern
/codeVersion-controlled scripts and Shiny application.Commit and sync to GitHub.
/dataABC Atlas cache and mounted read-only Data Assets.Cache may be deleted after capture; mounted Data Assets persist independently.
/resultsGenerated outputs from each processing stage.Capture important outputs as Data Assets.

Use /results for generated run outputs. Before closing a workstation, capture important files as Code Ocean Data Assets and mount those assets under /data for later stages. A captured Data Asset persists independently of the workstation.

Preflight checklist


2Convert and Format Back-end Files

Run create_dev_mouse_vis_cortex_pseudobulk.py

code/create_dev_mouse_vis_cortex_pseudobulk.py

Downloads the public Developing Mouse Visual Cortex data through AbcProjectCache, reads the raw count matrix in backed mode, and aggregates cells in chunks. The output has one row per library_label × cluster_alias, one column per gene, summed raw counts in X, sparse detection counts in layers["number_of_cells_expressing"], library/donor/taxonomy metadata in obs, and number_of_cells for each pseudobulk observation.

For every chunk, the aggregation computes both group_matrix @ count_chunk and group_matrix @ (count_chunk > 0). The second result is stored as unsigned integer counts and is the numerator for fraction expressing.

Use this run script:

#!/usr/bin/env bash
set -euo pipefail

mkdir -p /data/abc_atlas_cache
mkdir -p /results

python /code/create_dev_mouse_vis_cortex_pseudobulk.py \
  --cache-dir /data/abc_atlas_cache \
  --output /results/Developing_Mouse_Visual_Cortex_library_cluster_pseudobulk.h5ad

Run:

bash /code/run

Validate:

python - <<'PY'
import anndata as ad
path = "/results/Developing_Mouse_Visual_Cortex_library_cluster_pseudobulk.h5ad"
adata = ad.read_h5ad(path, backed="r")
print(adata.shape)
print(adata.obs.columns.tolist())
assert "library_label" in adata.obs.columns
assert "cluster_alias" in adata.obs.columns
assert "number_of_cells" in adata.obs.columns
assert "number_of_cells_expressing" in adata.layers
assert adata.layers["number_of_cells_expressing"].shape == adata.shape
assert adata.obs["number_of_cells"].min() > 0
adata.file.close()
print("Pseudobulk structure validation succeeded")
PY

Validate that all detection counts are between zero and the total number of contributing cells:

python - <<'PY'
import anndata as ad
import numpy as np
from scipy import sparse

path = "/results/Developing_Mouse_Visual_Cortex_library_cluster_pseudobulk.h5ad"
adata = ad.read_h5ad(path, backed="r")
detected = sparse.coo_matrix(adata.layers["number_of_cells_expressing"])
number_of_cells = adata.obs["number_of_cells"].to_numpy()
assert np.all(detected.data >= 0)
assert np.all(detected.data <= number_of_cells[detected.row])
adata.file.close()
print("Detection-count validation passed")
PY

Capture and mount the pseudobulk H5AD

  1. Verify the H5AD exists in the workstation results directory: ls -lh /root/capsule/results. If the workstation exposes the output only at /results, use readlink -f /results to identify the corresponding location.
  2. From the active Cloud Workstation interface, use the available results-capture control to create a Data Asset from /root/capsule/results. If that control is unavailable, generate a reproducible run and create the Data Asset from the result in the Capsule Timeline.
  3. Name it Developing Mouse Visual Cortex Pseudobulk with Detection Counts and describe X, number_of_cells_expressing, the grouping variables, and the originating script.
  4. Open the created Data Asset and verify that the H5AD is present before stopping the workstation.
  5. On the Capsule page, use the data-asset management panel to attach the asset. Use mount name mouse_vis_cortex_pseudobulk.
  6. Resume a fresh workstation and locate the mounted file with find /data -maxdepth 3 -type f -name "*.h5ad" -print.
find /data -type f -name "*.h5ad" -print

After the Data Asset has been validated, the temporary ABC Atlas cache can be removed:

rm -rf -- /data/abc_atlas_cache

Run create_mouse_aging_parquet.py

code/create_mouse_aging_parquet.py

Reads the mounted pseudobulk H5AD and writes the app-ready gene-level Parquet resource plus metadata. Each gene file contains sample_id, summed_counts, and unsigned-integer number_of_cells_expressing. The script validates the detection layer against number_of_cells and performs a Parquet read-back check. It does not create the statistics table; statistics are deliberately generated in the next independent step.

Run:

python /code/create_mouse_aging_parquet.py \
  --input "/data/mouse_vis_cortex_pseudobulk/Developing_Mouse_Visual_Cortex_library_cluster_pseudobulk.h5ad" \
  --output-dir "/results/mouse_vis_cortex_parquet"

Expected output:

/results/mouse_vis_cortex_parquet/
├── counts_by_gene/
│   ├── gene=<gene_key>/
│   │   └── data.parquet
│   └── ...
├── metadata/
│   ├── obs_metadata.parquet
│   ├── gene_map.parquet
│   └── CPM_scaling.csv
└── manifest.json

Run create_statistics.py

code/create_statistics.py

Reads the pseudobulk H5AD and generates the final exploratory statistics table independently of the Parquet conversion. Keeping this separate permits statistics to be revised without regenerating thousands of gene files.

Run:

python /code/create_statistics.py

The script writes:

/results/mouse_gene_summary_statistics.csv

Move or copy the generated file into the app-ready directory:

mkdir -p /results/mouse_vis_cortex_parquet/statistics
cp /results/mouse_gene_summary_statistics.csv \
  /results/mouse_vis_cortex_parquet/statistics/mouse_gene_summary_statistics.csv

The current table retains expression summaries and replaces the former (>0) pseudobulk metrics with true cell-detection summaries:

gene_symbol
ln(mean_CPM+1)
age_log2fc
frac_expr
age_frac_diff
max_ROI
log2fc_ROI
max_frac_ROI
frac_diff_ROI
max_subclass
log2fc_subclass
max_frac_subclass
frac_diff_subclass
gene_type

Keep the six short names synchronized between create_statistics.py and table_column_definitions.csv so the table tooltips remain accurate.

Run create_value_sets.py

code/create_value_sets.py

Creates a common value_sets.csv containing authoritative label order and color definitions. The script combines taxonomy annotation terms with dataset-level categorical value sets.

The script reads:

Run:

python /code/create_value_sets.py

Confirm the output is located at:

/results/mouse_vis_cortex_parquet/metadata/value_sets.csv

The normalized schema must be:

label,field,color_hex_triplet,order

Validate the complete back end

find /results/mouse_vis_cortex_parquet -type f | sort | head -30
find /results/mouse_vis_cortex_parquet -type f | wc -l
du -sh /results/mouse_vis_cortex_parquet

Verify key files:

python - <<'PY'
from pathlib import Path
import pandas as pd
import pyarrow.parquet as pq

root = Path("/results/mouse_vis_cortex_parquet")
required = [
    root / "metadata" / "obs_metadata.parquet",
    root / "metadata" / "gene_map.parquet",
    root / "metadata" / "CPM_scaling.csv",
    root / "metadata" / "value_sets.csv",
    root / "statistics" / "mouse_gene_summary_statistics.csv",
]
for path in required:
    assert path.exists(), path

metadata = pq.read_table(required[0]).to_pandas()
scaling = pd.read_csv(required[2])
assert len(metadata) == len(scaling)
assert metadata["library_label"].astype(str).tolist() == scaling["library_label"].astype(str).tolist()
gene_map = pq.read_table(required[1]).to_pandas()
assert metadata["sample_id"].is_unique
assert gene_map["gene_symbol"].is_unique
example = next((root / "counts_by_gene").glob("gene=*/data.parquet"))
columns = pq.read_schema(example).names
assert {"sample_id", "summed_counts", "number_of_cells_expressing"}.issubset(columns)
stats = pd.read_csv(required[4])
assert {"gene_symbol", "frac_expr", "age_frac_diff", "gene_type"}.issubset(stats.columns)
print("Back-end validation succeeded")
PY

Capture the complete app-ready back end

  1. Confirm the complete output is under /results/mouse_vis_cortex_parquet.
  2. Capture that directory as a second Code Ocean Data Asset for provenance and internal reuse.
  3. Open the asset and verify the counts_by_gene, metadata, and statistics directories.
  4. Attach it to any downstream Capsule that needs to inspect or republish the app-ready files.

3Publish Back-end Files to AWS

Configure the AWS CLI

Run AWS uploads from an interactive Code Ocean terminal using an approved AWS role or temporary credential for the target bucket. Never commit credentials to GitHub or save them in scripts, results, or Data Assets.

If the AWS CLI is not present, install it through the Code Ocean environment or current workstation, then verify:

aws --version
aws sts get-caller-identity

For temporary credentials:

export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_SESSION_TOKEN="..."
export AWS_DEFAULT_REGION="us-west-2"

aws sts get-caller-identity

Upload and verify the app-ready directory

Target prefix used by the application:

s3://sea-ad-single-cell-profiling/Multiregion_2026/previous_objects/parquet/TEST_dev_mouse/

Dry-run first:

aws s3 sync \
  "/results/mouse_vis_cortex_parquet/" \
  "s3://sea-ad-single-cell-profiling/Multiregion_2026/previous_objects/parquet/TEST_dev_mouse/" \
  --region us-west-2 \
  --dryrun

Upload:

aws s3 sync \
  "/results/mouse_vis_cortex_parquet/" \
  "s3://sea-ad-single-cell-profiling/Multiregion_2026/previous_objects/parquet/TEST_dev_mouse/" \
  --region us-west-2 \
  --progress-multiline \
  --progress-frequency 5
Do not add --delete to the real upload. Use it only with --dryrun for parity diagnostics.

Compare paths and sizes without changing AWS:

LOCAL="/results/mouse_vis_cortex_parquet"
S3="s3://sea-ad-single-cell-profiling/Multiregion_2026/previous_objects/parquet/TEST_dev_mouse"

aws s3 sync \
  "${LOCAL}/" \
  "${S3}/" \
  --region us-west-2 \
  --dryrun \
  --delete \
  --size-only

No output indicates matching paths and sizes. Compare counts and bytes:

find "${LOCAL}" -type f | wc -l
find "${LOCAL}" -type f -printf '%s\n' | awk '{t += $1} END {printf "%.0f\n", t}'

aws s3 ls "${S3}/" --recursive --summarize --region us-west-2 | tail -2

Confirm anonymous access because the deployed Shiny app does not use AWS credentials:

aws s3 ls \
  "${S3}/" \
  --no-sign-request \
  --region us-west-2

To overwrite the complete regenerated gene collection in its existing location while retaining unrelated remote objects:

LOCAL="/results/mouse_vis_cortex_parquet/counts_by_gene"
S3="s3://sea-ad-single-cell-profiling/Multiregion_2026/previous_objects/parquet/TEST_dev_mouse/counts_by_gene"
aws s3 sync "${LOCAL}/" "${S3}/" --region us-west-2 --dryrun
aws s3 sync "${LOCAL}/" "${S3}/" --region us-west-2 --only-show-errors

Do not use --delete for this production synchronization unless removal of remote-only files is explicitly intended.

Confirm that a representative public gene file contains the detection column:

aws s3 cp   "${S3}/gene=Reln/data.parquet"   /tmp/Reln_data.parquet   --no-sign-request   --region us-west-2
python - <<'PY'
import pyarrow.parquet as pq
path = "/tmp/Reln_data.parquet"
table = pq.read_table(path)
assert "number_of_cells_expressing" in table.column_names
print(table.schema)
PY

To replace only the statistics table after recalculation:

aws s3 cp \
  "/results/mouse_gene_summary_statistics.csv" \
  "${S3}/statistics/mouse_gene_summary_statistics.csv" \
  --region us-west-2

4Build and Test the Shiny Application

Application files

code/shiny/app.R

The complete R Shiny application. It loads lightweight metadata and statistics at startup, lazily retrieves selected gene directories from public S3, calculates CPM, caches retrieved genes, supports stacked categorical and numeric filters, and provides trajectory, heatmap, dot, violin, and two-gene correlation plots. Plot views include exploratory Spearman, Moran's I, Kruskal-Wallis, or Pearson statistics as appropriate.

code/shiny/abc_atlas_gene_url_function.R and abc_atlas_schema_kiwi.txt

The R function uses Node and kiwi-schema to replace the default Reln gene in a known working ABC Atlas state and return an otherwise equivalent gene-specific URL. app.R sources the function and rebuilds the sidebar link whenever a newly retrieved gene becomes loaded_gene().

code/shiny/table_column_definitions.csv

Provides local statistics-table header definitions using the required columns column_names and column_definitions. Keep this file beside app.R and include it in every deployment bundle.

code/shiny/www/

Static logos and browser assets. The current app requires Developing_Mouse_Visual_Cortex_logo.png and allen_institute_logo.svg. If tokens.csv exists with columns token, type, and name, the app displays an alphabetical searchable glossary; absent or invalid token files disable the glossary without stopping the app.

code/shiny/renv.lock

Records the tested R package versions used for local reconstruction and GitHub-driven Posit Connect deployment.

code/shiny/.Rprofile

Activates the project-local renv environment during development. Do not bundle this file in a Posit Connect deployment.

code/shiny/.rscignore

Excludes development-only files from the app bundle.

.Rprofile
.Rproj.user
.Rhistory
.RData
renv
renv/library
renv/staging
rsconnect
manifest.json

Do not exclude renv.lock.

How the app reads the published data

The app uses an anonymous Arrow S3 filesystem:

s3_bucket_name <- "sea-ad-single-cell-profiling"
s3_region <- "us-west-2"
s3_prefix <- paste0(
  "Multiregion_2026/previous_objects/parquet/",
  "TEST_dev_mouse"
)
s3_anonymous <- TRUE

At startup the app reads:

For a selected gene, it opens the entire directory:

counts_by_gene/gene=<gene_key>/

This permits one or multiple Parquet files per gene while avoiding discovery of the complete gene collection. The reader joins by sample_id, calculates CPM = summed_counts × CPM_scaling_factor, validates detection counts against number_of_cells, and calculates fraction_expressing. Retrieved genes are cached in memory.

The app derives stage at startup: donor ages beginning with E are labeled embryonic, and ages beginning with P are labeled postnatal. This app-only metadata field is then available for filtering, faceting, axes, and colors.

Configure and test gene-specific ABC Atlas links

  1. Place abc_atlas_gene_url_function.R and abc_atlas_schema_kiwi.txt beside app.R.
  2. Confirm app.R contains source("abc_atlas_gene_url_function.R").
  3. Confirm Node and kiwi-schema are available in every runtime environment, including Posit Connect.
  4. Test the function directly from code/shiny:
Rscript - <<'RS'
source("abc_atlas_gene_url_function.R")
url <- create_abc_atlas_gene_url(desired_gene = "Sst")
stopifnot(grepl("^https://knowledge.brain-map.org/abcatlas#", url))
cat(url, "
")
RS

In the app, Plot in ABC Atlas is rendered only after gene data are available. The URL is regenerated from the current loaded_gene() after each successful Get gene data action.

Preserve compact URL bookmarks

The app uses URL bookmarking but excludes ordinary Shiny and DataTable inputs dynamically. Only the compact custom viewer_state is saved. This prevents oversized URLs such as those caused by gene_statistics_table_rows_all.

Create and validate renv.lock

Generate the lockfile from a clean, known-working local R environment. Use R 4.3.3 and public repositories. Include the application packages plus rsconnect 

setwd("code/shiny")

options(
  repos = c(
    CRAN = "https://cloud.r-project.org"
  )
)

Sys.setenv(
  RENV_CONFIG_REPOS_OVERRIDE =
    "https://cloud.r-project.org"
)

packages_to_record <- c(
  "shiny",
  "bslib",
  "arrow",
  "dplyr",
  "ggplot2",
  "scales",
  "shinycssloaders",
  "cachem",
  "DT",
  "jsonlite",
  "rsconnect"
)

renv::snapshot(
  project = ".",
  library = .libPaths(),
  packages = packages_to_record,
  repos = c(CRAN = "https://cloud.r-project.org"),
  prompt = FALSE,
  force = TRUE
)

renv::status()

Verify that direct package versions in the lockfile equal the working library and that the lockfile does not contain codeocean.internal. The validated plotting stack uses ggplot2 3.5.2. Node.js and kiwi-schema are not recorded in renv.lock; record them separately in the Code Ocean environment and ensure the Posit Connect runtime also provides them.

Test locally

setwd("code/shiny")
renv::restore()
shiny::runApp(".")

Test:

Test in a Code Ocean RStudio workstation

Launch a fresh RStudio workstation after every environment rebuild. Test compiled package linkage before opening the application:

ldd /opt/conda/lib/R/library/arrow/libs/arrow.so | grep libstdc++
ldd /opt/conda/lib/R/library/curl/libs/curl.so | grep -E "ssl|crypto|curl"

Rscript -e 'library(arrow); library(curl); library(ggplot2); print(packageVersion("ggplot2"))'

Then run:

cd /root/capsule/code
Rscript -e 'shiny::runApp("shiny", host="0.0.0.0")'

5Deploy to Posit Connect

Perform the initial deployment

  1. Create a Posit Connect API key with publishing permission.
  2. Register https://molgen-shiny.org with rsconnect::addServer().
  3. Authenticate with rsconnect::connectApiUser().
  4. Deploy to a temporary test content item using strict lockfile resolution.
  5. After the test succeeds, deploy to the production content item.
setwd("code/shiny")

options(
  repos = c(
    CRAN = "https://cloud.r-project.org"
  )
)

Sys.setenv(
  RENV_CONFIG_REPOS_OVERRIDE =
    "https://cloud.r-project.org"
)

app_files <- c(
  "app.R",
  "abc_atlas_gene_url_function.R",
  "abc_atlas_schema_kiwi.txt",
  "table_column_definitions.csv",
  "renv.lock",
  list.files(
    "www",
    recursive = TRUE,
    full.names = TRUE
  )
)

rsconnect::deployApp(
  appDir = ".",
  appFiles = app_files,
  appPrimaryDoc = "app.R",
  appId = "3906a801-fab5-41e4-9c78-d7ce47cc0065",
  account = "<CONNECT_USERNAME>",
  server = "molgen-shiny",
  dependencyResolution = "strict",
  packageRepositoryResolutionR = "lax",
  forceUpdate = TRUE,
  launch.browser = FALSE,
  lint = TRUE,
  logLevel = "verbose"
)

Production application:

https://molgen-shiny.org/dev_mouse_gene_exploration/

Validate the Posit Connect deployment


6Configure GitHub Auto-deployment

Add the Posit Connect API key to GitHub

In the repository, open Settings → Secrets and variables → Actions and create:

CONNECT_API_KEY

Use a dedicated Posit Connect API key whose account has permission to update content GUID 3906a801-fab5-41e4-9c78-d7ce47cc0065. Never commit the key.

Create .github/workflows/deploy-connect.yml

.github/workflows/deploy-connect.yml

Runs after changes under code/shiny/, restores the exact R environment from renv.lock, generates a fresh strict deployment manifest, verifies that no private Code Ocean repository URL is present, and updates the existing Connect content.

name: Deploy Mouse Aging App to Posit Connect

on:
  push:
    branches:
      - main
    paths:
      - "code/shiny/**"
      - ".github/workflows/deploy-connect.yml"
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: deploy-mouse-developing-vis-app
  cancel-in-progress: true

jobs:
  deploy:
    runs-on: ubuntu-22.04

    env:
      RENV_PATHS_ROOT: ~/.cache/R/renv
      RENV_CONFIG_REPOS_OVERRIDE: https://cloud.r-project.org

    steps:
      - name: Check out repository
        uses: actions/checkout@v6

      - name: Set up R
        uses: r-lib/actions/setup-r@v2
        with:
          r-version: "4.3.3"
          use-public-rspm: true

      - name: Restore renv environment
        uses: r-lib/actions/setup-renv@v2
        with:
          working-directory: code/shiny
          cache-version: 2

      - name: Verify restored deployment environment
        shell: Rscript {0}
        run: |
          renv::load("code/shiny")
          stopifnot(requireNamespace("rsconnect", quietly = TRUE))
          cat("rsconnect:", as.character(packageVersion("rsconnect")), "\n")
          cat(paste(.libPaths(), collapse = "\n"), "\n")

      - name: Generate deployment manifest
        shell: Rscript {0}
        run: |
          renv::load("code/shiny")
          setwd("code/shiny")

          options(
            repos = c(
              CRAN = "https://cloud.r-project.org"
            )
          )

          Sys.setenv(
            RENV_CONFIG_REPOS_OVERRIDE =
              "https://cloud.r-project.org"
          )

          app_files <- c(
            "app.R",
            "abc_atlas_gene_url_function.R",
            "abc_atlas_schema_kiwi.txt",
            "table_column_definitions.csv",
            "renv.lock",
            list.files(
              "www",
              recursive = TRUE,
              full.names = TRUE
            )
          )

          rsconnect::writeManifest(
            appDir = ".",
            appFiles = app_files,
            appPrimaryDoc = "app.R",
            dependencyResolution = "strict",
            packageRepositoryResolutionR = "lax",
            verbose = TRUE
          )

          manifest_text <- paste(
            readLines("manifest.json", warn = FALSE),
            collapse = "\n"
          )

          stopifnot(
            !grepl(
              "codeocean.internal",
              manifest_text,
              fixed = TRUE
            )
          )

      - name: Verify deployment files
        shell: bash
        run: |
          test -f code/shiny/app.R
          test -f code/shiny/abc_atlas_gene_url_function.R
          test -f code/shiny/abc_atlas_schema_kiwi.txt
          test -f code/shiny/table_column_definitions.csv
          test -f code/shiny/renv.lock
          test -f code/shiny/manifest.json
          test -f code/shiny/www/allen_institute_logo.svg
          test -f code/shiny/www/Developing_Mouse_Visual_Cortex_logo.png
          test -f code/shiny/www/tokens.csv

      - name: Deploy to Posit Connect
        uses: posit-dev/connect-actions/deploy@main
        with:
          connect-server: https://molgen-shiny.org
          connect-api-key: ${{ secrets.CONNECT_API_KEY }}
          content-guid: 3906a801-fab5-41e4-9c78-d7ce47cc0065
          path: code/shiny
          draft: false

After committing the workflow, monitor the first run under the repository's Actions tab. The workflow can also be run manually because it includes workflow_dispatch.

Update R dependencies safely

  1. Make package changes in a clean local project environment.
  2. Run the application and complete functional tests.
  3. Run renv::snapshot() with the explicit package list, including rsconnect.
  4. Run renv::status() and verify critical installed and locked versions match.
  5. Commit renv.lock with the app change.
  6. Push to a branch and, if desired, test against a temporary Connect content item before merging to main.
Why rsconnect is locked: The GitHub workflow calls rsconnect::writeManifest() from the restored project library, so rsconnect and its recursive dependencies must be present in renv.lock even though the Shiny runtime does not call them.

7Publish Documentation with GitHub Pages

Publish index.html and this guide

  1. Place index.html and deployment_documentation.html at the repository root.
  2. In GitHub, open Settings → Pages.
  3. Select main and /(root) as the publishing source.
  4. Commit and push both HTML files.
  5. Add a link from index.html to this guide if desired.

Because the repository is private, GitHub may assign a unique private Pages subdomain. Routine deployments update the content at that same private Pages address.


Maintenance Checklist

When only app.R or CSS changes

  1. Test locally.
  2. Commit changes under code/shiny/.
  3. Push or merge to main.
  4. Confirm the GitHub Action succeeds.
  5. Validate the live app.

When an R package changes

  1. Update the local project library.
  2. Test all application functionality.
  3. Run the explicit renv::snapshot().
  4. Commit both code and renv.lock.
  5. Review the Connect build log after deployment.

Commit and deploy future application updates

After editing and testing the application locally, review the changed files, commit them, and push the branch. Changes under code/shiny/ trigger the Posit Connect deployment workflow after they reach main; root documentation updates are published through the configured GitHub Pages source.

git status
git diff

git add code/shiny/app.R \
  code/shiny/table_column_definitions.csv \
  code/shiny/renv.lock \
  index.html \
  deployment_documentation.html

git commit -m "Update mouse gene expression viewer"
git push origin <branch-name>

If only some files changed, include only those files in git add. Do not commit manifest.json, local renv libraries, generated scientific data, credentials, or ignored runtime artifacts.

After pushing, merge the tested branch to main using the repository's normal review process. Confirm the GitHub Actions deployment succeeds and validate the live app. For a direct update from main:

git checkout main
git pull --ff-only origin main
git status
git add <changed-files>
git commit -m "Describe the application update"
git push origin main
Before pushing: Run the local regression checklist, verify that table_column_definitions.csv accompanies app.R, and update renv.lock only when the tested package environment changed.

When source data or preprocessing changes

  1. Regenerate the pseudobulk H5AD if the aggregation definition changed.
  2. Capture the revised H5AD as a versioned Data Asset.
  3. Regenerate gene Parquet and metadata.
  4. Regenerate statistics independently.
  5. Regenerate value_sets.csv if taxonomy, anatomical labels, order, or colors changed.
  6. Validate all files and capture the complete output as a Data Asset.
  7. Upload to a new AWS test prefix.
  8. Test the app against the test prefix.
  9. Promote the tested prefix or update s3_prefix in app.R.
  10. Push the app change to trigger deployment.

Troubleshooting

SymptomLikely causeResolution
object 'counts_dataset' not foundThe global dataset was removed for faster startup but read_gene() still references it.Open the selected counts_by_gene/gene=<gene_key>/ directory inside read_gene(); do not open the full collection at startup.
First gene is as slow as startup used to beA delayed global dataset still discovers all gene partitions.Remove the delayed global dataset and open only the selected gene directory.
CXXABI_1.3.15 not foundArrow loads an older system libstdc++.Install current Conda C++ runtimes and place /opt/conda/lib first in LD_LIBRARY_PATH.
libssl.so.1.1 not found for R curlAn old R curl binary was compiled against OpenSSL 1.1.Install current libcurl/openssl and build the R curl package against that environment.
Incompatible methods ("Ops.S7_object", "+.gg")Incompatible ggplot2/S7 package stack.Use the validated ggplot2 3.5.2 environment.
Library and lockfile are out of syncThe active R library differs from renv.lock.Rebuild and test a clean library, then snapshot the tested package versions. Use strict resolution only after synchronization.
there is no package called 'rsconnect' in GitHub ActionsThe project library was restored but not activated in the standalone Rscript step, or rsconnect was omitted from the lockfile.Record rsconnect in renv.lock and call renv::load("code/shiny") before using it.
Connect cannot resolve codeocean.internalThe manifest captured Code Ocean's private package proxy.Set public repositories before manifest generation and fail the workflow if the private URL appears.
invalid 'width' or 'height' in plottingPlot output or parent containers report a transient invalid dimension.Use one explicit responsive sizing model, allow containers to shrink with min-height: 0, and avoid competing height calculations.
Windows shell script reports invalid pipefailCRLF line endings.Convert code/run to LF and enforce line endings with .gitattributes.
414 Request-URI Too LargeAutomatic bookmarking serialized DataTable and dynamic-input state, especially all table row indices.Dynamically exclude all ordinary inputs and store only the compact custom viewer_state.
Operation not allowed without an active reactive context during restorationA reactive value was read outside a reactive consumer.Wrap the read in isolate(), and keep dynamic filter restoration state until inputs confirm their values.
The first plot requires two clicksLazy eventReactive() evaluation races with plot-tab activation.Capture settings into a reactiveVal() first, then select the plot tab in session$onFlushed().
Plot continuously regenerates after resizingexecOnResize = TRUE reruns the complete expression while reactive height and scrollbar geometry change.Use execOnResize = FALSE; regenerate only after Generate plot or bookmark restoration.
LOESS warnings about singularities or failed groupsSome color/facet groups have too few distinct progression values for stable LOESS fitting.These warnings concern smoothing groups; use a linear fit or no fit for sparse configurations, and first ensure resize-triggered redraw loops are disabled.
Facet page errors such as missing TRUE/FALSE or argument length zeroThe dynamically created page input is briefly NULL.Default missing page size to 12 and missing or invalid page number to 1 before any if condition.
The ABC Atlas button keeps the first geneThe URL was calculated only for the first click or stored in a nonreactive browser handler.Regenerate a reactive URL whenever loaded_gene() changes and render the link from that value.
kiwi-schema cannot be foundNode cannot resolve the globally installed npm module.Install kiwi-schema with npm and set NODE_PATH=/opt/conda/lib/node_modules.

References

Rebuild criterion: A successful rebuild ends with validated summed-count and detection-count pseudobulk matrices, captured Code Ocean provenance assets, gene-level Parquet files with detection counts, current statistics and value sets, an anonymously readable public S3 back end, a locally and remotely tested Shiny application, working bookmarks and ABC Atlas links, a reproducible renv.lock, a working Posit Connect content item, and a GitHub Action that updates that content whenever code/shiny/ changes.