Pipeline Logs and Artifacts but Locally

For yourself and others, retaining records of what happened when deploying changes to cloud infrastructure is super handy. Adverse downstream effects from changes or outfall from state-surgery often don’t rear their ugly heads until minutes, hours, or even days later. Being a built-in feature of CI/CD pipelines, these historical records are a godsend when all hell breaks loose, or the teammate with some costly tribal knowledge left the team. Both to help your recollection and also to get your team involved.

Built on open standards such as Markdown, OpenTelemetry, Web Components, and the VS Code platform.

Automated the Unautomatable

Runme notebooks are a hit when automating the last mile as pre-pipeline automation. The last mile we understand as anything running locally, close to a human, instructions not uniform enough to run in a pipeline, requiring the explorative gathering of input information to complete a sequence of steps or processes subject to frequent changes. Anything where maintaining CI/CD pipelines turns into a fool's errand trying to chase moving targets. And, in case targets no longer move, you can always stick your Runme notebook into a CI/CD pipeline.

Session Outputs in action using a terraform example.

Unlocking Session Outputs

Like with CI/CD pipelines, we know how powerful logs and build artifacts are. This is why, today, in Runme’s v3.0 release, we are introducing the Session Outputs feature for notebooks. The idea is simple yet powerful. When enabled (as in Auto-save (on)), Runme will capture a full copy of the original Markdown document with all cell outputs generated by running them in the notebook UX as an artifact alongside the original file.

Outputs file rollout-01HP7QRAVY5W9N5NGR25V9FZ2W.md in the same directory as rollout.md.

Every cell output includes when it ran, time elapsed, and exit codes. More contextual metadata, e.g., hostname, user name, etc, could be saved to the Session Outputs file. Runme will also capture non-text mime types like images (base64 encoded) outside of textual output produced by terminals. Feel free to let us know if you have feedback.

Cell with input and session output.

Session Outputs are written per Runme session, which, unless reset via Reset Session in the notebook UX, encompasses the lifetime of the window that’s running the notebook. Providing other session lifecycle choices such as notebook open & close or time window (threshold of elapsed time between running commands, e.g., >30min) is possible. Let us know if you have specific requirements in mind.

Notebook UX to control session. Click reset to start a new session.

Adjust Your Settings

The notebook’s default is for Session Outputs capturing to be off (Auto-Save (off)) in the top bar. Whenever required, it’s as easy as clicking the Auto-Save button to toggle the feature (on) or (off). However, changing the Session Outputs default to enable “Auto-Save” by default using an option in settings is possible. In case you don’t want to worry about manually enabling it every time.

💡 If you’re logged into the Cloud (prompt whenever Auto-Save is toggled on), cells will both be stored locally in a Session Outputs file and in your dashboard for later review or sharing with others. This is entirely optional and, if desired, can be turned off (follow “Open Settings” in prompt).

Search @ext:stateful.runme runme.app.notebookAutoSave in your settings.

Why Separate Output Files

If you are asking yourself why we would choose to write a separate Session Outputs file instead of writing outputs back into the markdown document, we do have reasons:

• We believe cells and their contained code & commands (inputs) should be generic to enable collaboration and sharing of notebooks with others. We wanted to maintain this property, especially for version control tracked files.
• The file separation also gets ahead of accidentally checking in outputs into version control and accidental overwriting of historical session artifacts if they reside in the same file.
• Session files with outputs won’t open in the notebook UX. Runme won’t be able to discern what parts are input or output belonging to a cell after the output is written. The UX will prompt to open the original document in the same folder or launch the markdown previewer for Session Outputs.

While we’re considering adding transparent opening of Session Outputs files in the future, it will almost inevitably require reformatting the source file as part of de-/serialization. However, if and when this becomes a priority on Runme’s roadmap, it will most likely be an optional feature. If you have thoughts or ideas about this, please get in touch.

Outside of capturing outputs, we are also working on adding emitting OpenTelemetry traces to standard-compliant collectors into Runme. This will allow Runme users to put output artifacts into the context of time and location, harnessing broadly available OpenTelemetry tooling. Curious how your team could benefit from Runme? Click here to learn more.

Changes in Runme v3.0

If you have used prior versions of Runme, the changes in v3 are relatively subtle. Here's what you need to know:

• Runme's version number in the markdown file's metadata will no longer store major and minor digits, e.g., v3.0. It will store only the major number v3. Please see Runme v2 for details.
• Running cells, i.e., generating cell output, will now cause the notebook editor to mark the document as modified (dot next to the filename in the editor tab) and require saving. Unless you modify input cells, this will, in most cases, not affect the source file. However, the session outputs file will receive updates.
• The order of the annotation configuration options in the Configure foldout was reshuffled based on their frequency of usage.
• You can altogether opt-out of the Session Outputs feature via the @ext:stateful.runme runme.app.sessionOutputs setting, which is default enabled.

Runme in a Nutshell

If you haven’t already, check out the video we made about Runme below. It does a terrific job explaining the core ideas behind it. If you have questions or feedback, please let us know.

Enjoy and tell all your friends! See you on Discord.