Toolchain · 31 tools and approaches
The tools,
and the thinking
behind them.
Good tools make room for the work. These are the languages, libraries and design choices behind the applications, data systems and research — what they help with, and the trade-offs they introduce. Some belong to working products; others are part of a research design.
31
Tools and approaches
7
Areas of the build
9
Languages across the work
22
Projects to explore
01 — Languages
A good fit for each part of the work
Different languages support research, data, interfaces and performance-sensitive code. Clear boundaries keep each part understandable and the connections between them maintainable.
-
Python
Research, data and automationIn practice The main language across the analysis and pipeline work. Its scientific libraries make it practical to move from an exploratory notebook to a repeatable process.
Considerations A bottleneck can be optimised or moved into a native extension while the surrounding workflow stays readable.
-
Rust
Native applicationsIn practice Used in the native application work for control over memory, concurrency and rendering. It brings the application shell closer to the graphics and system tools it relies on.
Considerations Native development brings more platform work. It fits projects where responsiveness and system integration justify that investment; web interfaces suit work that benefits from browser access.
-
C++
Numerical hot pathsIn practice A way to accelerate focused numerical operations, such as rolling indicators, while exposing them to Python through a small interface.
Considerations The extra build and integration work needs a clear payoff. Most application and research logic is easier to change in the higher-level layer.
-
R
Statistical researchIn practice Used for econometrics, statistical exploration and model comparison. It lets the research draw on established packages and methods.
Considerations A separate runtime needs a clear interface and reproducible inputs. Keeping that boundary small makes the analysis easier to rerun and review.
-
JavaScript · TypeScript
Websites and applicationsIn practice JavaScript supports focused browser interfaces; TypeScript helps organise applications with richer data and shared contracts. Both connect the analysis to an experience people can use.
Considerations The build setup should match the project. A small site and a growing application have different needs for frameworks, types and reusable components.
-
SQL · WGSL · Shell
Queries, graphics and automationIn practice SQL makes data transformations explicit, WGSL supports GPU graphics work, and shell scripts connect repeatable development tasks.
Considerations Libraries and abstractions can save time. Direct access to the underlying layer remains useful when a query, shader or operational task needs closer control.
02 — Machine learning
Start with the data available
The sensor research explores learning from limited labels. Document projects use retrieval and review to connect generated text with source material. Each calls for a different balance of modelling and engineering.
-
PyTorch + Lightning
Training and experimentsResearch tooling A shared framework for model experiments, training loops and checkpoints. Lightning helps keep routine training behaviour consistent across runs.
Considerations A standard framework saves setup time, while custom loops remain useful when an experiment needs more control.
-
Self-supervised pretraining
Sensor representationsResearch design Explored in Ichne to learn useful patterns from unlabelled sensor streams before adapting the model to a smaller labelled dataset.
Considerations Its value needs to be checked against simpler baselines and the data available for the intended task.
-
QLoRA on a compact model
Task-specific language outputResearch design A proposed way to adapt a compact language model within a limited compute budget, while leaving most of the pretrained weights unchanged.
Considerations Model size, training data and output quality need to be tested together. An adapter is a useful option, rather than a guarantee of a better result.
-
llama.cpp · quantised weights
Local inferenceDesign option Part of the local and batch inference approach, where smaller models can help keep processing close to the data and reduce infrastructure needs.
Considerations Hardware, throughput and response-time requirements determine whether CPU inference, a local GPU or a hosted service is the better fit.
-
Retrieval before generation
Document workflowsIn practice Relevant source material is collected and ranked before a model drafts a response. Reviewers can see what the answer is based on.
Considerations Retrieval does not remove the need for review. Source coverage, ranking quality and checks on the final output all affect the result.
03 — Estimation and geometry
Turn measurements into a coherent picture
The trajectory research combines learned motion estimates with explicit geometric constraints. Calculations and uncertainty stay available for inspection alongside the visual result.
-
Factor-graph smoothing
Trajectory reconstructionResearch design Combines motion, heading, floor and reference-point information in one estimation problem. A batch workflow can use later observations to refine earlier parts of a trajectory.
Considerations Smoothing needs more history and computation than an online estimate. The balance depends on whether the result is needed during a shift or afterwards.
-
Learned motion uncertainty
Sensor modellingResearch design The proposed motion model predicts both displacement and uncertainty, allowing the estimator to weight more ambiguous segments differently.
Considerations The uncertainty estimates need calibration. A simple noise model remains a useful baseline for checking whether the learned version adds value.
-
Deterministic vector rendering
Maps and diagramsIn practice Maps and diagrams can be drawn from structured geometry, keeping the displayed result connected to the coordinates and values behind it.
Considerations Raster imagery is useful context. Measured geometry stays in a separate structured layer so it can still be checked and reused.
04 — Quantitative systems
Connect research with the running system
The trading tools aim to keep strategy logic, data handling and risk controls consistent across historical research and execution. That makes differences easier to find and investigate.
-
Shared strategy logic
Research and executionIn practice Strategy code is shared between replay and the live path, reducing the amount of logic that has to be implemented and checked twice.
Considerations Shared code still needs checks for differences in data arrival, fees, fills and broker behaviour. Reuse makes those checks more focused.
-
Event-driven replay
BacktestingIn practice Historical events pass through the same kind of interface as incoming data, making timing and information availability explicit in the research workflow.
Considerations Vectorised analysis remains useful for fast exploration. Event replay is used when sequencing and execution assumptions need closer attention.
-
Layered position sizing
Risk controlsIn practice Strategy signals feed a separate sizing and risk layer, with limits, drawdown controls and cooldowns represented explicitly in the system.
Considerations A more flexible sizing rule adds assumptions to test. Fixed-size and fixed-fraction approaches provide useful comparison points.
-
Watchdogs and stop controls
Operational safeguardsIn practice Dedicated controls handle stalled processes, manual stops and recovery paths. They make failure behaviour part of the application design.
Considerations Their behaviour must be tested against the broker connection and available state. An attempted close or stop needs confirmation and a visible record.
05 — Data and storage
A foundation people can keep using
The data work spans public-source collection, healthcare datasets, operating models and application state. Storage choices follow the shape of the data, access needs and ongoing operating cost.
-
PostgreSQL
Relational applicationsIn practice Used where records, relationships and transactional updates need a common home. SQL also makes many transformations visible and reviewable in one place.
Considerations Some workflows fit a small local database or a document store. The choice depends on access patterns and the application around the data.
-
Database-backed jobs
Background workDesign option A jobs table can keep work status close to the records it affects and give a small application a straightforward operational view.
Considerations A dedicated queue becomes useful when delivery guarantees, scheduling, throughput or service boundaries need more specialised behaviour.
-
Serverless storage
Intermittent workloadsDesign option Considered for workloads with long idle periods, where pausing or scaling down the infrastructure can fit the way the system is used.
Considerations Idle savings need to be weighed against wake-up time, service limits, geographic features and the cost of moving data between services.
-
Dataset and model versioning
ReproducibilityIn practice Keeping datasets, model artifacts and experiment settings connected makes it easier to identify the inputs behind a result and revisit an earlier run.
Considerations Versioning needs a clear retention and storage policy. The goal is to preserve useful lineage without keeping every intermediate file indefinitely.
06 — Interfaces and delivery
Bring the work into everyday use
Websites, dashboards, native tools and generated reports each serve a different kind of work. The delivery format depends on how people need to read, explore or act on the result.
-
Static generation
Websites and publishingIn practice Used for this site and structured publishing workflows. Content is assembled into pages at build time, giving it consistent navigation and presentation.
Considerations A CMS or application backend becomes useful when editing, permissions or frequently changing data need to happen outside the build process.
-
Queues and workers
Long-running analysisIn practice Document and data jobs can run in the background with visible status, retries and an output to return to when the work is finished.
Considerations Background processing adds state to manage. Short tasks can stay in a request; longer work benefits from explicit progress and recovery.
-
Native GPU interfaces
Desktop applicationsIn practice Used in native tool development to explore responsive charts, terminals and graphics within a dedicated application shell.
Considerations Browser interfaces offer easier distribution and broad access. Native rendering is a project-specific choice when the workflow benefits from closer system integration.
-
Model Context Protocol
AI and developer toolsIn practice Connects tools such as workbook inspection and data access to compatible assistants through a common interface.
Considerations The protocol is only part of the integration. Permissions, validation, useful errors and reliable tool behaviour still need careful implementation.
-
Structured research publishing
Reports and researchIn practice Markdown chapters, source registers and chart specifications feed a repeatable publishing process for linked HTML and PDF outputs.
Considerations Automation helps keep long documents consistent. Editorial review and visual checks are still needed for page flow, charts and readable explanations.
07 — Quality and maintenance
Make the next iteration easier
Validation, testing and operational feedback help a useful prototype become a dependable tool. The checks focus on the ways a project is likely to change or fail.
-
Validated data boundaries
Data qualityIn practice Checks at important handoffs catch missing fields, unexpected units and changed source formats before they spread through a pipeline.
Considerations Validation rules need to evolve with the source. Useful errors and review paths help distinguish a bad record from a legitimate change.
-
Structured logging
TroubleshootingIn practice Job, record and stage identifiers make it easier to follow a process and locate the part that needs attention.
Considerations Useful logs provide enough context to diagnose a problem while keeping sensitive data out of routine output.
-
Property-based testing
Numerical and geometry codeTesting approach Generated inputs can check useful invariants across a wider range of cases, complementing examples drawn from real workflows.
Considerations Example-based tests remain valuable for known failures and expected behaviour. The two approaches answer different questions.
-
Resumable stages
Data pipelinesIn practice Long processes record progress so a failed stage can be retried without repeating all the work that came before it.
Considerations Resuming safely requires clear rules for saved state and repeated writes. Those rules are part of the pipeline design.
Built around the problem
The right tools make
the next step easier.
Every project has its own constraints. Tool choices depend on the people using the result, the data available and what it will take to maintain. The case studies show those choices in context, including what is built and what is still being explored.