State inspection script

Asset Tracker Template

tags
Asset Tracker Template

The inspect_state.py script lets you inspect the current state of the application's state machines and internal data structures on a running device. It supports the following two modes of operation:

  • Live inspection: The script connects to the device using J-Link, parses the ELF file to find the symbol locations and types, and reads the memory to display the current state.
  • Coredump analysis: The script can also analyze a coredump file that the device generated, which lets you inspect the state at the time of the crash.

The script is particularly useful for debugging when the application is stuck or behaves unexpectedly and you want to see the exact state of each module without halting the CPU or adding extensive logging, and when you analyze crashes.

The script has the following prerequisites:

  • Python 3 installed.
  • The pyelftools package, version 0.30 or later, and the pylink-square package:

    pip install "pyelftools>=0.30" pylink-square
    
  • A J-Link debug probe connected to the device.

  • The zephyr.elf file that corresponds to the running firmware.

Run the script from the scripts directory, or adjust the path, and provide the path to your ELF file and optionally the J-Link device name:

# Live inspection
python3 Asset-Tracker-Template/scripts/inspect_state.py --elf build/app/zephyr/zephyr.elf
# Coredump analysis
python3 Asset-Tracker-Template/scripts/inspect_state.py --elf path/to/symbols.elf --coredump path/to/coredump.elf

The script first shows a summary table of all modules and their current state:

Connecting to J-Link (Cortex-M33)...

...

Module          | Current State                                                | Details
----------------------------------------------------------------------------------------------------
Main            | STATE_CONNECTED_WAITING                                      | Ptr: 0x00094D80
Cloud           | STATE_CONNECTED_READY                                        | Ptr: 0x000966F0
Location        | STATE_LOCATION_SEARCH_INACTIVE                               | Ptr: 0x00096628
Network         | STATE_CONNECTED                                              | Ptr: 0x00094E70
FOTA            | STATE_WAITING_FOR_POLL_REQUEST                               | Ptr: 0x000967A8
Env             | STATE_RUNNING                                                | Ptr: 0x000965BC
Power           | STATE_RUNNING                                                | Ptr: 0x00094F68
Storage         | STATE_BUFFER_IDLE                                            | Ptr: 0x00096848

Options:
  q: Quit
  r: Refresh summary
  1: Inspect Main
  2: Inspect Cloud
  3: Inspect Location
  4: Inspect Network
  5: Inspect FOTA
  6: Inspect Env
  7: Inspect Power
  8: Inspect Storage

Select option: 2

Selecting a module, for example 2 for the Cloud module, reveals the detailed structure of its state variable, including all internal members:

--- Cloud State Details ---
Address: 0x0008C8C0
Type: cloud_state
----------------------------------------
ctx                 : STATE_CONNECTED_READY
chan                : 0x0008BE5C
msg_buf             : Array[584] (too large to display)
network_connected   : 1 (0x1)
provisioning_ongoing: 0 (0x0)
connection_attempts : 2 (0x2)
backoff_time        : 60 (0x3C)
----------------------------------------