Labgrid Jetson

A labgrid driver package for NVIDIA Jetson development kit bootstrap, power control, and boot orchestration.

Overview

This package provides specialized drivers for NVIDIA Jetson development kits:

• JetsonBootstrapDriver: Complete bootstrap/flash operations implementing BootstrapProtocol
• JetsonPowerDriver: Power control with Jetson-specific timing and toggle support
• JetsonStrategy: Boot strategy for coordinating flash and boot sequences

All drivers use standard labgrid protocols for maximum compatibility and remote operation support.

Installation

pip install labgrid-jetson

For development:

git clone <repository>
cd labgrid-jetson
pip install -e .[dev]  # Installs package + dev dependencies (pytest, build, twine, etc.)

Drivers

JetsonBootstrapDriver

Complete bootstrap driver that handles recovery mode entry, flashing, and boot.

Implements: BootstrapProtocol

Protocol Dependencies:

• PowerProtocol - Device power control
• DigitalOutputProtocol - Recovery pin control

Features:

• Automated recovery mode entry (power off → assert pin → power on → de-assert pin)
• Tegraflash archive extraction and flashing
• Supports both .tar.zst and standard tar archives
• Real-time flash progress logging
• Automatic boot after flash completes
• Remote operation compatible
• Configurable timing parameters

Configuration Example:

drivers:
  # Power control
  power:
    cls: JetsonPowerDriver
    name: power
    initial_state: false
    bindings:
      output: power_output

  # Recovery pin control
  recovery_output:
    cls: SomeDigitalOutputDriver
    name: recovery_output
    bindings:
      channel: recovery_pin

  # Bootstrap driver
  bootstrap:
    cls: JetsonBootstrapDriver
    name: bootstrap
    archive: /path/to/tegraflash.tar.zst  # Optional - can be provided programmatically
    recovery_setup_delay: 0.5
    recovery_boot_delay: 2.0
    power_off_delay: 0.5
    bindings:
      power: power
      recovery: recovery_output

Note: The archive parameter is optional in the configuration. You can omit it and provide the archive path programmatically to load() instead.

Usage:

from labgrid import Environment
from labgrid.util.managedfile import ManagedFile

# Load environment
env = Environment("jetson.yaml")
target = env.get_target("jetson-devkit")

# Get bootstrap driver
bootstrap = target.get_driver("JetsonBootstrapDriver")

# Option 1: Flash with configured archive
bootstrap.load()

# Option 2: Flash with different archive (overrides configured archive)
bootstrap.load("/path/to/different/archive.tar.zst")

# Option 3: Flash with programmatically fetched archive
import subprocess
subprocess.run(["scp", "server:/releases/latest.tar.zst", "/tmp/archive.tar.zst"])
bootstrap.load("/tmp/archive.tar.zst")

# Option 4: Flash with ManagedFile (transparent remote sync)
# ManagedFile automatically syncs file from client to exporter
mf = ManagedFile("/local/client/path/archive.tar.zst", bootstrap)
bootstrap.load(mf)  # Automatically syncs and uses remote path
# After flashing, device automatically reboots to normal mode

# Manual recovery mode control (for advanced use cases)
# Note: flash operations automatically exit recovery mode
bootstrap.enter_recovery_mode()  # Device is now in recovery mode
# ... perform custom recovery operations ...
bootstrap.exit_recovery_mode()   # Powers off, de-asserts pin, powers on

Recovery Mode Sequence:

The driver implements the standard Jetson recovery sequence:

Enter Recovery Mode:

1. Power off device (ensures clean state)
2. Assert recovery pin (HIGH)
3. Power on device (boots into recovery mode)
4. De-assert recovery pin (only needed during boot) → Device is now in recovery mode, ready for flashing

Flash Operation:

• Extract tegraflash archive
• Run initrd-flash script (5-15 minutes depending on image)
• Flash process automatically reboots device to normal mode

Exit Recovery Mode (manual, only if not flashing):

1. Power off device
2. De-assert recovery pin
3. Power on device → Device boots normally

Methods:

• load(filename=None): Complete bootstrap/flash operation (BootstrapProtocol). Device automatically reboots to normal mode after flash.
• enter_recovery_mode(): Put device into recovery mode (power off → assert pin → power on → de-assert pin)
• exit_recovery_mode(): Manually exit recovery mode and boot normally (power off → de-assert pin → power on)

Parameters:

• archive (str, optional): Path to tegraflash archive. If not provided in configuration, must be passed to load() method.
• recovery_setup_delay (float): Delay after asserting recovery before power on (default: 0.5s)
• recovery_boot_delay (float): Delay after power on while in recovery mode (default: 2.0s)
• power_off_delay (float): Delay after power off before asserting recovery (default: 0.5s)

Archive Handling: The archive can be specified in four ways:

1. Static configuration: Set archive parameter in YAML configuration
2. Runtime override: Pass path to load(filename) to override configured archive
3. Dynamic only: Omit archive in config, always provide to load(filename)
4. ManagedFile: Pass a ManagedFile object for transparent client-to-exporter sync

ManagedFile Support (for remote operation): When working with labgrid's remote architecture (coordinator/exporter), use ManagedFile to automatically sync archives from client to exporter:

from labgrid.util.managedfile import ManagedFile

# File on client machine
mf = ManagedFile("/local/path/archive.tar.zst", bootstrap)
bootstrap.load(mf)  # Automatically syncs to exporter and uses remote path

The driver automatically detects ManagedFile objects and:

• Calls sync_to_resource() to copy file to exporter
• Uses get_remote_path() to get the exporter-side path
• Logs sync progress

This flexibility is useful for CI/CD pipelines, release testing, or fetching archives from remote servers at runtime.

JetsonPowerDriver

Power control driver with Jetson-specific timing and toggle-based power control.

Implements: PowerProtocol

Protocol Dependencies:

• DigitalOutputProtocol - Power pin control

Features:

• Toggle-based power control with state tracking
• Jetson-specific timing (5s for off, 0.5s for on)
• Prevents double-toggles via state management
• Requires initial_state configuration
• Configurable pulse durations

Configuration Example:

drivers:
  power:
    cls: JetsonPowerDriver
    name: power
    initial_state: false  # REQUIRED - observe actual device state
    power_off_duration: 5.0  # Optional, default 5.0s
    power_on_duration: 0.5   # Optional, default 0.5s
    bindings:
      output: power_output  # Any DigitalOutputProtocol

Usage:

from labgrid import Environment

env = Environment("jetson.yaml")
target = env.get_target("jetson-devkit")
power = target.get_driver("JetsonPowerDriver")

# Standard PowerProtocol methods
power.on()     # Power on (0.5s pulse)
power.off()    # Power off (5s pulse)
power.cycle()  # Power cycle (off then on)

# Get current tracked state
state = power.get()  # Returns True (on) or False (off)

Important: The initial_state parameter is required. You must observe the actual device power state and configure it accordingly. The power pin toggles state rather than directly controlling it.

Methods:

• on(): Power on device (only toggles if currently off)
• off(): Power off device (only toggles if currently on)
• cycle(): Power cycle device (off then on)
• get(): Get tracked power state (True=on, False=off)

Parameters:

• initial_state (bool): REQUIRED - Initial power state (True=on, False=off)
• power_off_duration (float): Duration to assert for power off (default: 5.0s)
• power_on_duration (float): Duration to assert for power on (default: 0.5s)

JetsonStrategy

High-level boot strategy that orchestrates flash and boot sequences with smart power management.

Implements: labgrid Strategy pattern

Protocol Dependencies:

• PowerProtocol - Device power control
• ResetProtocol (optional) - Reset control
• SerialDriver - Serial communication
• ShellDriver - Shell access
• BootstrapProtocol - Flashing operations

Features:

• Smart power cycle: checks current state and cycles if on, powers on if off
• Automatic fallback to bootstrap if power cycle fails
• Optional reset-based boot instead of power cycle
• Configurable boot timeout and post-flash wait
• Force flash mode to skip power cycle and go straight to bootstrap
• Programmatic archive selection that overrides environment configuration

Configuration Example:

drivers:
  # All prerequisite drivers...
  power:
    cls: JetsonPowerDriver
    # ... power configuration

  reset:
    cls: DigitalOutputResetDriver
    # ... reset configuration (optional)

  bootstrap:
    cls: JetsonBootstrapDriver
    archive: /path/to/default/image.tar.zst
    # ... bootstrap configuration

  serial:
    cls: SerialDriver
    # ... serial configuration

  shell:
    cls: ShellDriver
    # ... shell configuration

  # Strategy - orchestrates everything
  strategy:
    cls: JetsonStrategy
    boot_timeout: 120.0        # Time to wait for boot and shell (default: 120s)
    force_flash: false         # Skip power cycle, go straight to flash (default: false)
    post_flash_wait: 30.0      # Wait after flash before shell (default: 30s)
    use_reset: false           # Use reset instead of power cycle (default: false)
    archive: /path/to/override/image.tar.zst  # Optional: override bootstrap archive

Usage:

from labgrid import Environment

env = Environment("jetson.yaml")
target = env.get_target("jetson-devkit")
strategy = target.get_driver("JetsonStrategy")

# Standard transition to shell - tries power cycle first, falls back to flash
strategy.transition("shell")

# Or force flash (skip power cycle attempt)
strategy.force_flash = True
strategy.transition("shell")

# Programmatically select flash archive
strategy.archive = "/releases/test-build-v1.2.3.tar.zst"
strategy.force_flash = True
strategy.transition("shell")

# Turn off device
strategy.transition("off")

State Machine:

The strategy manages two states:

• off: Device powered off
• shell: Device booted with shell access

Transition from off → shell:

1. Try power cycle boot (unless force_flash=True):

   • Check current power state
   • If on: cycle power (off → on)
   • If off: power on
   • Wait for shell with boot_timeout
   • If successful: done ✓
   • If failed: proceed to step 2

2. Bootstrap (flash) and boot:

   • Activate bootstrap driver
   • If strategy.archive is set: pass to bootstrap.load(archive)
   • Otherwise: use bootstrap's configured archive
   • Wait post_flash_wait seconds
   • Establish shell connection
   • Done ✓

Parameters:

• boot_timeout (float): Seconds to wait for boot and shell after power cycle (default: 120.0)
• force_flash (bool): Skip power cycle and go straight to bootstrap (default: False)
• post_flash_wait (float): Seconds to wait after flash before establishing shell (default: 30.0)
• use_reset (bool): Use reset instead of power cycle for boot (default: False)
• archive (str, optional): Override the bootstrap driver's configured archive path

Dynamic Archive Selection:

The archive parameter allows runtime control of which image to flash:

# CI/CD: Test different images
test_images = [
    "/releases/minimal-v1.tar.zst",
    "/releases/full-v1.tar.zst",
    "/releases/debug-v1.tar.zst"
]

for image in test_images:
    strategy.archive = image
    strategy.force_flash = True
    strategy.transition("shell")
    # ... run tests ...
    strategy.transition("off")

# Release testing: Fetch latest build
import subprocess
subprocess.run(["scp", "build-server:/releases/latest.tar.zst", "/tmp/test.tar.zst"])
strategy.archive = "/tmp/test.tar.zst"
strategy.force_flash = True
strategy.transition("shell")

When to Use Each Mode:

• Power cycle first (force_flash=False, default):

  • Quick iteration during development
  • Device already has correct image flashed
  • Testing boot behavior or kernel changes
  • Recovery from soft failures

• Force flash (force_flash=True):

  • CI/CD pipelines testing new images
  • First-time device setup
  • Known boot failures requiring reflash
  • Switching between different images

• Reset-based boot (use_reset=True):

  • Hardware has reset button wired
  • Reset is more reliable than power cycle
  • Faster than power cycle

Complete Configuration Examples

Local Configuration

imports:
  - labgrid_jetson

targets:
  main:
    resources:
      # Hardware resources (example using GPIO)
      power_pin:
        cls: SomeDigitalOutputResource
        pin: 23

      recovery_pin:
        cls: SomeDigitalOutputResource
        pin: 24

      reset_pin:
        cls: SomeDigitalOutputResource
        pin: 25

    drivers:
      # Low-level digital output drivers
      power_output:
        cls: SomeDigitalOutputDriver
        name: power_output
        bindings:
          resource: power_pin

      recovery_output:
        cls: SomeDigitalOutputDriver
        name: recovery_output
        bindings:
          resource: recovery_pin

      reset_output:
        cls: SomeDigitalOutputDriver
        name: reset_output
        bindings:
          resource: reset_pin

      # Jetson power driver
      power:
        cls: JetsonPowerDriver
        name: power
        initial_state: false  # Set based on actual device state
        bindings:
          output: power_output

      # Reset driver
      reset:
        cls: DigitalOutputResetDriver
        name: reset
        bindings:
          output: reset_output

      # Jetson bootstrap driver
      bootstrap:
        cls: JetsonBootstrapDriver
        name: bootstrap
        archive: /path/to/tegraflash.tar.zst
        bindings:
          power: power
          recovery: recovery_output

      # Serial and shell drivers
      serial:
        cls: SerialDriver

      shell:
        cls: ShellDriver
        prompt: 'root@.*:~# '
        login_prompt: 'login: '
        username: 'root'

      # Strategy for complete orchestration
      strategy:
        cls: JetsonStrategy
        boot_timeout: 120.0
        force_flash: false
        post_flash_wait: 30.0
        use_reset: false

Remote Configuration (via Coordinator)

imports:
  - labgrid_jetson
  - labgrid_webrelaycon  # For network relay control

targets:
  main:
    resources:
      RemotePlace:
        name: my-jetson-place  # Place name from coordinator

    drivers:
      # WebRelayCon drivers - bind to exported resources
      power_output:
        cls: WebRelayConDriver
        name: power_output
        bindings:
          channel: my-jetson-power-pin  # Resource name from exporter

      recovery_output:
        cls: WebRelayConDriver
        name: recovery_output
        bindings:
          channel: my-jetson-recovery-pin

      reset_output:
        cls: WebRelayConDriver
        name: reset_output
        bindings:
          channel: my-jetson-reset-pin

      # Jetson power driver
      power:
        cls: JetsonPowerDriver
        name: power
        initial_state: false
        bindings:
          output: power_output

      # Reset driver
      reset:
        cls: DigitalOutputResetDriver
        name: reset
        bindings:
          output: reset_output

      # Jetson bootstrap driver
      bootstrap:
        cls: JetsonBootstrapDriver
        name: bootstrap
        archive: /path/to/tegraflash.tar.zst  # Path on exporter machine
        bindings:
          power: power
          recovery: recovery_output

      # Serial driver - automatically binds to NetworkSerialPort
      serial:
        cls: SerialDriver

      # Shell driver
      shell:
        cls: ShellDriver
        prompt: 'root@.*:~# '
        login_prompt: 'login: '
        username: 'root'

      # Strategy
      strategy:
        cls: JetsonStrategy
        boot_timeout: 120.0
        force_flash: false
        post_flash_wait: 30.0
        use_reset: false

Supported Jetson Devices

The recovery sequence is standardized across all Jetson development kits:

• Jetson Nano
• Jetson Xavier NX
• Jetson AGX Xavier
• Jetson Orin Nano
• Jetson Orin NX
• Jetson AGX Orin

Remote Operation

All drivers are designed for labgrid's remote operation via coordinator/exporter architecture.

Architecture

[Client Machine]                [Exporter Machine]
    pytest                          labgrid-exporter
      ↓                                   ↓
  Environment                    Hardware Resources:
      ↓                          - Jetson USB (recovery)
  Coordinator  ←─────────────→  - Serial port
      ↓                          - Network relays (power, recovery, reset)
  RemotePlace                           ↓
      ↓                          Exported as NetworkResources
  Drivers (remote proxies)

How It Works

1. Exporter Setup: Hardware resources (USB, serial, relays) are exported via labgrid-exporter and registered with the coordinator
2. Client Configuration: Test configuration uses RemotePlace to reference the exporter's resources
3. Resource Binding: Drivers bind to remote resources (NetworkSerialPort, WebRelayConChannel, etc.)
4. Command Execution: Driver commands are sent through the coordinator to the exporter
5. Flash Operations: Archive extraction and flashing happen on the exporter (where USB is connected)
6. Log Streaming: Real-time flash progress is streamed back to the client

Key Points

• Archive path: Must be accessible on the exporter machine (not the client)
• USB access: Jetson USB for recovery mode must be connected to the exporter
• Serial port: Can be NetworkSerialPort (ser2net) or direct USB serial
• Relay control: Uses WebRelayConDriver or similar for remote GPIO/relay control
• Flash execution: Runs on exporter where USB device is accessible
• Logging: All output streams to client in real-time

Running Tests Remotely

# Run test via coordinator
PYTHONPATH=/path/to/drivers pytest -v -s \
  test_strategy.py \
  --lg-env=remote-config.yaml \
  --lg-coordinator=hostname:20408

# With real-time logging
PYTHONPATH=/path/to/drivers pytest -v -s \
  --log-cli-level=INFO \
  test_strategy.py \
  --lg-env=remote-config.yaml \
  --lg-coordinator=hostname:20408

Testing

Configuration Validation

Test your configuration without flashing:

python3 -c "
from labgrid import Environment
env = Environment('your-config.yaml')
target = env.get_target('main')
bootstrap = target.get_driver('JetsonBootstrapDriver')
print(f'Bootstrap driver loaded: {bootstrap}')
print(f'Archive: {bootstrap.archive}')
print(f'Power: {bootstrap.power}')
print(f'Recovery: {bootstrap.recovery}')
"

Pytest Integration

The strategy works seamlessly with pytest and labgrid's target fixture:

# test_jetson.py
def test_boot_and_verify(target):
    """Test boot and verify system information."""
    from labgrid_jetson import JetsonStrategy
    from labgrid.driver import ShellDriver

    # Get drivers
    strategy = target.get_driver(JetsonStrategy)
    shell = target.get_driver(ShellDriver, activate=False)

    # Ensure clean state
    strategy.transition("off")

    # Boot device
    strategy.transition("shell")

    # Run verification commands
    stdout, stderr, returncode = shell.run("uname -a")
    assert returncode == 0
    print(f"System: {stdout}")

    stdout, stderr, returncode = shell.run("cat /etc/os-release")
    assert returncode == 0
    print(f"OS: {stdout}")

def test_flash_new_image(target):
    """Test flashing a specific image."""
    from labgrid_jetson import JetsonStrategy

    strategy = target.get_driver(JetsonStrategy)

    # Override archive for this test
    strategy.archive = "/releases/test-build.tar.zst"
    strategy.force_flash = True

    # Flash and boot
    strategy.transition("shell")

    # Verify flash succeeded
    # ... run tests ...

def test_power_cycle_recovery(target):
    """Test power cycle boot without reflashing."""
    from labgrid_jetson import JetsonStrategy

    strategy = target.get_driver(JetsonStrategy)
    strategy.force_flash = False  # Try power cycle first

    # This will try power cycle, fallback to flash if needed
    strategy.transition("shell")

Run tests:

# Local
pytest -v test_jetson.py --lg-env=jetson.yaml

# Remote via coordinator
PYTHONPATH=/path/to/drivers \
  pytest -v -s test_jetson.py \
  --lg-env=remote-config.yaml \
  --lg-coordinator=hostname:20408

Interactive Testing

Use labgrid-client for interactive control:

# Acquire place
labgrid-client -x hostname:20408 -p my-jetson-place acquire

# Enter recovery mode
labgrid-client -x hostname:20408 -p my-jetson-place bootstrap enter-recovery

# Flash device
labgrid-client -x hostname:20408 -p my-jetson-place bootstrap load /path/to/archive.tar.zst

# Access serial console
labgrid-client -x hostname:20408 -p my-jetson-place console

# Release place
labgrid-client -x hostname:20408 -p my-jetson-place release

Protocol Architecture

This package follows labgrid's protocol-based architecture:

JetsonBootstrapDriver (implements BootstrapProtocol)
├── bindings.power (PowerProtocol)
│   └── JetsonPowerDriver
│       └── bindings.output (DigitalOutputProtocol)
└── bindings.recovery (DigitalOutputProtocol)

JetsonPowerDriver (implements PowerProtocol)
└── bindings.output (DigitalOutputProtocol)

This allows for:

• Hardware independence
• Easy testing and mocking
• Remote operation support
• Flexible composition

Troubleshooting

Power state out of sync

If the power state gets out of sync, the device may be in the wrong state. To fix:

1. Observe the actual device power state (LED, serial console, etc.)
2. Update initial_state in configuration to match
3. Restart your labgrid client/test

Flash fails with "device not found"

The device may not be in recovery mode. Check:

1. Recovery pin is properly connected and controlled
2. Power sequence completed successfully
3. USB connection between Jetson and flash host
4. Wait longer with recovery_boot_delay parameter

Timeouts during flash

Flashing can take 5-30 minutes depending on image size. The driver has a 1-hour timeout. If you hit this:

1. Check USB connection stability
2. Verify archive is not corrupted
3. Check available disk space on flash host

Development and Publishing

Installing Development Dependencies

Install the package in editable mode with development tools (pytest, build, twine, etc.):

pip install -e .[dev]

Building and Publishing

The project includes a Makefile for building and uploading packages:

make help          # Show available targets
make clean         # Remove build artifacts
make build         # Build wheel and source distribution
make upload-gitea  # Build and upload to Gitea registry
make upload-pypi   # Build and upload to PyPI

Publishing Configuration

Create ~/.pypirc with your registry credentials:

[distutils]
index-servers =
    pypi
    gitea

[pypi]
username = __token__
password = <your-pypi-token>

[gitea]
repository = https://gitea.konsulko.com/api/packages/mporter/pypi
username = mporter
password = <your-gitea-token>

Security: Ensure ~/.pypirc has restricted permissions:

chmod 600 ~/.pypirc

Then publish with:

make upload-gitea  # Upload to Gitea
# or
make upload-pypi   # Upload to PyPI

Note: The version in setup.py must be incremented before uploading a new release, as registries don't allow overwriting existing versions.

License

This project follows the same license as labgrid.

Contributing

Contributions are welcome! Please ensure:

• Code follows existing style
• All drivers use standard labgrid protocols
• Documentation is updated
• Tests pass

Version

Current version: 1.0.1