Toolchain · 31 tools and approaches

The tools,
and the thinking
behind them.

Good tools make room for the work. These are the languages, libraries and design choices behind the applications, data systems and research — what they help with, and the trade-offs they introduce. Some belong to working products; others are part of a research design.

31

Tools and approaches

7

Areas of the build

9

Languages across the work

22

Projects to explore

01 — Languages

A good fit for each part of the work

Different languages support research, data, interfaces and performance-sensitive code. Clear boundaries keep each part understandable and the connections between them maintainable.

  • Python

    Research, data and automation

    In practice The main language across the analysis and pipeline work. Its scientific libraries make it practical to move from an exploratory notebook to a repeatable process.

    Considerations A bottleneck can be optimised or moved into a native extension while the surrounding workflow stays readable.

  • Rust

    Native applications

    In practice Used in the native application work for control over memory, concurrency and rendering. It brings the application shell closer to the graphics and system tools it relies on.

    Considerations Native development brings more platform work. It fits projects where responsiveness and system integration justify that investment; web interfaces suit work that benefits from browser access.

  • C++

    Numerical hot paths

    In practice A way to accelerate focused numerical operations, such as rolling indicators, while exposing them to Python through a small interface.

    Considerations The extra build and integration work needs a clear payoff. Most application and research logic is easier to change in the higher-level layer.

  • R

    Statistical research

    In practice Used for econometrics, statistical exploration and model comparison. It lets the research draw on established packages and methods.

    Considerations A separate runtime needs a clear interface and reproducible inputs. Keeping that boundary small makes the analysis easier to rerun and review.

  • JavaScript · TypeScript

    Websites and applications

    In practice JavaScript supports focused browser interfaces; TypeScript helps organise applications with richer data and shared contracts. Both connect the analysis to an experience people can use.

    Considerations The build setup should match the project. A small site and a growing application have different needs for frameworks, types and reusable components.

  • SQL · WGSL · Shell

    Queries, graphics and automation

    In practice SQL makes data transformations explicit, WGSL supports GPU graphics work, and shell scripts connect repeatable development tasks.

    Considerations Libraries and abstractions can save time. Direct access to the underlying layer remains useful when a query, shader or operational task needs closer control.

02 — Machine learning

Start with the data available

The sensor research explores learning from limited labels. Document projects use retrieval and review to connect generated text with source material. Each calls for a different balance of modelling and engineering.

  • PyTorch + Lightning

    Training and experiments

    Research tooling A shared framework for model experiments, training loops and checkpoints. Lightning helps keep routine training behaviour consistent across runs.

    Considerations A standard framework saves setup time, while custom loops remain useful when an experiment needs more control.

  • Self-supervised pretraining

    Sensor representations

    Research design Explored in Ichne to learn useful patterns from unlabelled sensor streams before adapting the model to a smaller labelled dataset.

    Considerations Its value needs to be checked against simpler baselines and the data available for the intended task.

  • QLoRA on a compact model

    Task-specific language output

    Research design A proposed way to adapt a compact language model within a limited compute budget, while leaving most of the pretrained weights unchanged.

    Considerations Model size, training data and output quality need to be tested together. An adapter is a useful option, rather than a guarantee of a better result.

  • llama.cpp · quantised weights

    Local inference

    Design option Part of the local and batch inference approach, where smaller models can help keep processing close to the data and reduce infrastructure needs.

    Considerations Hardware, throughput and response-time requirements determine whether CPU inference, a local GPU or a hosted service is the better fit.

  • Retrieval before generation

    Document workflows

    In practice Relevant source material is collected and ranked before a model drafts a response. Reviewers can see what the answer is based on.

    Considerations Retrieval does not remove the need for review. Source coverage, ranking quality and checks on the final output all affect the result.

03 — Estimation and geometry

Turn measurements into a coherent picture

The trajectory research combines learned motion estimates with explicit geometric constraints. Calculations and uncertainty stay available for inspection alongside the visual result.

  • Factor-graph smoothing

    Trajectory reconstruction

    Research design Combines motion, heading, floor and reference-point information in one estimation problem. A batch workflow can use later observations to refine earlier parts of a trajectory.

    Considerations Smoothing needs more history and computation than an online estimate. The balance depends on whether the result is needed during a shift or afterwards.

  • Learned motion uncertainty

    Sensor modelling

    Research design The proposed motion model predicts both displacement and uncertainty, allowing the estimator to weight more ambiguous segments differently.

    Considerations The uncertainty estimates need calibration. A simple noise model remains a useful baseline for checking whether the learned version adds value.

  • Deterministic vector rendering

    Maps and diagrams

    In practice Maps and diagrams can be drawn from structured geometry, keeping the displayed result connected to the coordinates and values behind it.

    Considerations Raster imagery is useful context. Measured geometry stays in a separate structured layer so it can still be checked and reused.

04 — Quantitative systems

Connect research with the running system

The trading tools aim to keep strategy logic, data handling and risk controls consistent across historical research and execution. That makes differences easier to find and investigate.

  • Shared strategy logic

    Research and execution

    In practice Strategy code is shared between replay and the live path, reducing the amount of logic that has to be implemented and checked twice.

    Considerations Shared code still needs checks for differences in data arrival, fees, fills and broker behaviour. Reuse makes those checks more focused.

  • Event-driven replay

    Backtesting

    In practice Historical events pass through the same kind of interface as incoming data, making timing and information availability explicit in the research workflow.

    Considerations Vectorised analysis remains useful for fast exploration. Event replay is used when sequencing and execution assumptions need closer attention.

  • Layered position sizing

    Risk controls

    In practice Strategy signals feed a separate sizing and risk layer, with limits, drawdown controls and cooldowns represented explicitly in the system.

    Considerations A more flexible sizing rule adds assumptions to test. Fixed-size and fixed-fraction approaches provide useful comparison points.

  • Watchdogs and stop controls

    Operational safeguards

    In practice Dedicated controls handle stalled processes, manual stops and recovery paths. They make failure behaviour part of the application design.

    Considerations Their behaviour must be tested against the broker connection and available state. An attempted close or stop needs confirmation and a visible record.

05 — Data and storage

A foundation people can keep using

The data work spans public-source collection, healthcare datasets, operating models and application state. Storage choices follow the shape of the data, access needs and ongoing operating cost.

  • PostgreSQL

    Relational applications

    In practice Used where records, relationships and transactional updates need a common home. SQL also makes many transformations visible and reviewable in one place.

    Considerations Some workflows fit a small local database or a document store. The choice depends on access patterns and the application around the data.

  • Database-backed jobs

    Background work

    Design option A jobs table can keep work status close to the records it affects and give a small application a straightforward operational view.

    Considerations A dedicated queue becomes useful when delivery guarantees, scheduling, throughput or service boundaries need more specialised behaviour.

  • Serverless storage

    Intermittent workloads

    Design option Considered for workloads with long idle periods, where pausing or scaling down the infrastructure can fit the way the system is used.

    Considerations Idle savings need to be weighed against wake-up time, service limits, geographic features and the cost of moving data between services.

  • Dataset and model versioning

    Reproducibility

    In practice Keeping datasets, model artifacts and experiment settings connected makes it easier to identify the inputs behind a result and revisit an earlier run.

    Considerations Versioning needs a clear retention and storage policy. The goal is to preserve useful lineage without keeping every intermediate file indefinitely.

06 — Interfaces and delivery

Bring the work into everyday use

Websites, dashboards, native tools and generated reports each serve a different kind of work. The delivery format depends on how people need to read, explore or act on the result.

  • Static generation

    Websites and publishing

    In practice Used for this site and structured publishing workflows. Content is assembled into pages at build time, giving it consistent navigation and presentation.

    Considerations A CMS or application backend becomes useful when editing, permissions or frequently changing data need to happen outside the build process.

  • Queues and workers

    Long-running analysis

    In practice Document and data jobs can run in the background with visible status, retries and an output to return to when the work is finished.

    Considerations Background processing adds state to manage. Short tasks can stay in a request; longer work benefits from explicit progress and recovery.

  • Native GPU interfaces

    Desktop applications

    In practice Used in native tool development to explore responsive charts, terminals and graphics within a dedicated application shell.

    Considerations Browser interfaces offer easier distribution and broad access. Native rendering is a project-specific choice when the workflow benefits from closer system integration.

  • Model Context Protocol

    AI and developer tools

    In practice Connects tools such as workbook inspection and data access to compatible assistants through a common interface.

    Considerations The protocol is only part of the integration. Permissions, validation, useful errors and reliable tool behaviour still need careful implementation.

  • Structured research publishing

    Reports and research

    In practice Markdown chapters, source registers and chart specifications feed a repeatable publishing process for linked HTML and PDF outputs.

    Considerations Automation helps keep long documents consistent. Editorial review and visual checks are still needed for page flow, charts and readable explanations.

07 — Quality and maintenance

Make the next iteration easier

Validation, testing and operational feedback help a useful prototype become a dependable tool. The checks focus on the ways a project is likely to change or fail.

  • Validated data boundaries

    Data quality

    In practice Checks at important handoffs catch missing fields, unexpected units and changed source formats before they spread through a pipeline.

    Considerations Validation rules need to evolve with the source. Useful errors and review paths help distinguish a bad record from a legitimate change.

  • Structured logging

    Troubleshooting

    In practice Job, record and stage identifiers make it easier to follow a process and locate the part that needs attention.

    Considerations Useful logs provide enough context to diagnose a problem while keeping sensitive data out of routine output.

  • Property-based testing

    Numerical and geometry code

    Testing approach Generated inputs can check useful invariants across a wider range of cases, complementing examples drawn from real workflows.

    Considerations Example-based tests remain valuable for known failures and expected behaviour. The two approaches answer different questions.

  • Resumable stages

    Data pipelines

    In practice Long processes record progress so a failed stage can be retried without repeating all the work that came before it.

    Considerations Resuming safely requires clear rules for saved state and repeated writes. Those rules are part of the pipeline design.

Built around the problem

The right tools make
the next step easier.

Every project has its own constraints. Tool choices depend on the people using the result, the data available and what it will take to maintain. The case studies show those choices in context, including what is built and what is still being explored.