1 Establish a Clear Vision

You are a human, you know how this world behaves, how your team and colleagues behave, and what your users expect. You have experienced the world, and you want to work together with a system that has no experience in this world you live in. Every decision in your project that you don’t take and document will be taken for you by the AI.

Your responsibility of delivering quality code cannot be met if not even you know where long-lasting and difficult-to-change decisions are taken.
You must know what parts of your code need to be thought through and what must be vigorously tested.

Think about and discuss the architecture, interfaces, data structures, and algorithms you want to use. Think about how to test and validate your code to these specifications.

2 Maintain Precise documentation

You need to communicate to the AI in detail what you want to achieve, otherwise it will result in code that is unusable for your purpose.

Other developers also need to communicate this information to the AI. That makes it efficient to write as much documentation as practical in a standardized format and into the code repository itself.

Document the requirements, specifications, constraints, and architecture of your project in detail.
Document your coding standards, best practices, and design patterns.
Use flowcharts, UML diagrams, and other visual aids to communicate complex structures and workflows.
Write pseudocode for complex algorithms and logic to guide the AI in understanding your intentions.

3 Build debug systems that aid the AI

Develop efficient debug systems for the AI to use, reducing the need for multiple expensive CLI commands or browsers to verify code functionality. This will save time and resources while simplifying the process for the AI to identify and resolve code issues.

For example: Build a system that collects logs from all nodes in a distributed system and provides abstracted information like “The Data was send to all nodes”, “The Data X is saved on Node 1 but not on Node 2”.

4 Mark code review levels

Not all code is equally important. Some parts of your codebase are critical and need to be reviewed with extra care. Other parts are less important and can be generated with less oversight.

Use a system that allows you to mark how thoroughly each function has been reviewed.

For example you can use a prompt that will let the AI put the comment //A behind functions it wrote to indicate that the function has been written by an AI and is not yet reviewed by a human.

5 Write high level specifications and test by yourself

AIs will cheat and use shortcuts eventually. They will write mocks, stubs, and hard coded values to make the code tests succeed while the code itself is not working and most of the time dangerous. Often AIs will adapt or outright delete test code to let the code pass tests.

You must discourage this behavior by writing property based high level specification tests yourself. Build them in a way that makes it hard for the AI to cheat without having big code segments dedicated to it.
For example, use property based testing, restart the server and check in between if the database has the correct values.

Separate these test so the AI cannot edit them and prompt the AI not to change them.

6 Write interface tests in a separate context

Let an AI write property based interface tests for the expected behavior with as little context of the rest of the code as possible.
This will generate tests that are uninfluenced by the “implementation AI” which will prevent the tests from being adapted to the implementation in a way that makes them useless or less effective.

Separate these tests so the AI cannot edit them without approval and prompt the AI not to change them.

7 Use strict linting and formatting rules

Use strict linting and formatting rules to ensure code quality and consistency. This will help you and your AI to find issues early.

8 Use context specific coding agent prompts

Save time and money by utilizing path specific coding agent prompts like CLAUDE.md.

You can generate them automatically which will give your AI information it would otherwise as to create from scratch every time.

Try to provide as much high level information as practical, such as coding standards, best practices, design patterns, and specific requirements for the project. This will help the AI to generate code that is more aligned with your expectations and will reduce lookup time and cost.

9 Find and mark functions that have a high security risk

Identify and mark functions that have a high security risk, such as authentication, authorization, and data handling. These functions should be reviewed and tested with extra care and in such a way that a human has comprehended the logic of the function in all its dimensions and is confident about its correctness and safety.

Make this explicit with a comment like //HIGH-RISK-UNREVIEWED and //HIGH-RISK-REVIEWED to make sure that other developers are aware of the importance of these functions and will review them with extra care.

Make sure that the AI is instructed to change the review state of these functions as soon as it changes a single character in the function.
Developers must make sure that the status of these functions is always correct.

10 Reduce code complexity where possible

Aim to reduce the complexity of the generated code where possible. Each single line of code will eat up your context window and make it harder for the AI and You to keep track of the overall logic of your code.
Each avoidable line of code is costing energy, money and probability of future unsuccessful AI tasks.

11 Explore problems with experiments and prototypes

AI written code is cheap, use this to your advantage by exploring different solutions to a problem with experiments and prototypes with minimal specifications. This will allow you to find the best solution to a problem without investing too much time and resources in a single solution.

12 Do not generate blindly or to much complexity at once

Break down complex tasks into smaller, manageable tasks for the AI. Instead of asking the AI to generate the complete project or component at once, break it down into smaller tasks, such as generating individual functions or classes. This will help you to maintain control over the code and it’s logic.

You have to check each component or module for its adherence to the specifications and requirements.
If you have lost the overview of the complexity and inner workings of the code, you have lost control over your code and must restart from a state where you were in control of your code.

As i coded on OuroborosDB i noticed that i need a very fast queue for a rather unique architectural design decision.
I try to build the network module in such a way that i can test the behavior completely deterministic while “simulating” entire clusters in a single process.

So i build a test prototype of my global network queue with go’s channels and noticed that it was a major performance bottleneck, after writing 2 different ring buffer queue implementations it became clear that some queues behave completely different under different congestion levels and core counts - some so unpredictable that i just did not wanted to use them in my project.

This prompted me to take a relatively large chunk out of my free time and write a suite to benchmark different queue implementations i build under different conditions and score them based on their performance and predictability.

The result of this work is GoQueueBench

These are the results of the benchmark suite:

Please note that i build the package so that all queue adhere to the same interface and can be swapped out easily.

I want to share this great article about generic coding in Go: https://bitfieldconsulting.com/posts/constraints

Also Go Proverbs is quite nice!

Ohh and this Blog has now footnotes and popups for footnotes! ¹

Summary (Generated):

In this tutorial, John Arundel explores constraints in Go, the paradoxical feature of generics that simultaneously limits and expands possibilities. By delving into method sets, type elements, unions, intersections, and approximations, he demonstrates how constraints enable precise operations on type parameters. Whether you’re learning to leverage fmt.Stringer, creating flexible numeric constraints, or handling derived types, this guide unlocks a deeper understanding of Go’s generics—perfect for honing your skills or tackling practical challenges in coding.

I forgot to mention that i released a new tool for my project ouroboros-db, so here it is:

The “OuroborosDB Overhead Calculator” is a little tool which helps you to calculate the overhead of erasure coding systems while also considering the overhead of indexes and the blocks themselves. It is kinda fun to play around with so give it a try!

https://i5heu.github.io/ouroboros-db-overhead-calculator/

This is a Dev Journal for the Ouroboros DB Project.
I try to write down my thoughts and ideas Somewhat structured to have it as a reference for later and i publish it to give this information a chance to help other and to get feedback from the community.
If you have feedback you can write me on my Mastodon.

Pls note that this is only a Dev Journal and not a Blog Post, so it may be a bit unstructured and not as polished as a Blog Post, including typos and other errors.

TL;DR:

Today i worked on a refactoring of the architecture and feasibility of erasure coding for Ouroboros DB. There are some concerns i had regarding the potential index size and overhead that will result from it, and as it turns out, it is not as bad as i thought but i need a DHT for the index.

Architecture

I think it would be super handy to be able to run parts of data pipelines (eg. storing files into chunked,compressed, encrypted and then erasure coded blocks) in a distributed manner. for safety and because i want to be able to run trustless nodes that can’t see the raw data, it would be necessary to have the file chunking, compression and encryption part on the server the client currently speaks too, although it would be possible to pre-chunk the file on the client and upload it to different nodes, although here is the question if it is easy to port the chunking algorithm to browser JS or WASM.

Modules

Refactoring of the code is needed to implement erasure coding, maybe we can even get rid of the chunk entirely as a stored thing since we have the data in the parity blocks already.

Maybe best to add the erasure coding to the StoreDataPipeline and add the needed erasure coding metadata to ChunkData.

ASCII Art of the new architecture (click to get the .txt file):

Erasure Coding

For now i think it is okay yo make the erasure coding a simple n6 k3, which would result in a parity block size of 327,680 Bytes or 0.31MB as average. This would result in about 1.875MB per chunk that is 1.25MB big in average. This would result in a 50% overhead for the erasure coding, which i think is quite okay.

For the goal of storing 100TB in it, which are about 400M Chunks, we would need 2,400,000,000 parity blocks, aka 2,4 Billion.

If we consider following overhead for the erasure coding:

{
"parityHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
"chunkHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
"sizeByte": 433000,
"lastChecked": 1724760126282,
"reblanceLog": [
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    }
],
"userLog": [
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    }
],
"this is for the KV Key": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0"
}

We see that parity block metadata has under 2373 Bytes of overhead, which is about 0.72%% of the parity block size (very good). If one node would need to store the entire hash table we look at a size of 5.18TB, which means that we need a DHT for this to work. This also is the case for the Events, which introduce more storage overhead (more in DHT )

DHT

We need additional DHT Metadata like this:

{
  "parityHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
  "storingNodes": [
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    }
  ]
}

This example meta data has a size of 674 Bytes, which is about 0.21% of the parity block size. The DHT meta data would require an additional 1.47TB of storage.

Overhead

DHT meta data in combination with the parity block meta data would have a total overhead of 0.93%% of the erasure coding metadata relative to the erasure coding parity block size, the entire metadata overhead at 100TB and 2,4 Billion parity blocks would be 6.65TB.

Adding this to the overhead from the erasure coding which are 50% we would have a total overhead of 56.65% relative to the raw data size. If we have a utilization of each chunk of 1310720 Bits which has been achieved with 10MB files with random binary data. This would result in a HDD to raw data ratio of 43.35% (which is pretty good for a k6n3 config and having self healing capabilities) for following configuration:

• 100TB of raw data
• data deduplication via BuzzHash into chunks
• chunk compression via Zstd
• chunk encryption via AES256
• each chunk becomes 6 parity blocks of which 3 can be lost without data loss

Conclusion

I have written a Google Calc Spreadsheet for it, which you can find here: ouroboros-db Overhead Calculator.

Though this Spreadsheet is neat, i discoverd that i need some surface plotting to find the best configuration for the erasure coding.
Sadly there is no way to do this in Google Calc, so i will need to write a small Browser App for it… maybe i use Svelte for it, i haven’t used it in a while now.

Further Reading

A good overview over what erasure coding is about: Erasure Coding for Distributed Systems
Also See HN comments

In the midst of working on my project, ouroboros-db, I realized I needed a straightforward way to locate other nodes using the same RSA key. That’s when PathfinderBeacon came into being and that i release to the public with these words.

What is PathfinderBeacon?

PathfinderBeacon is a tool designed to make it easier to bootstrap your distributed systems using just a RSA key and one of the most robust dictionaries available: DNS. Since DNS inherently caches information at various levels, it allows PathfinderBeacon to deliver node addresses quickly and with minimal bandwidth usage. This means if multiple searches are conducted within the same network for the same room, there’s a high chance the requests won’t even leave your local network, thanks to the caching capabilities of your router’s local DNS.

You can think of it like a DDNS (Dynamic Domain Name System) service, but you only need an RSA key to write to a subdomain. The subdomain is generated from the RSA key, and there is little surface for abuse since you can only publish highly sanitized TXT records.

How It Works

Setting up is pretty simple. You either provide an RSA key or let PathfinderBeacon generate one for you. Once set, you can publish your room and node addresses, and anyone with the same RSA key can write to the same room, while anyone can read from it. This makes it easy to bootstrap your distributed systems without the need of running your own bootstrap service.

Currently, there is an official Golang library for interacting with PathfinderBeacon, which you can find here. If you’re interested in developing libraries for other programming languages, feel free to reach out, and I’d be glad to assist and subsequently feature your work on the project repository.

Future Directions

For me this project is pretty much finished since it solves the underlying problem quite effectively, but i am fascinated by this problem and i would love to develop PathfinderBeacon further, maybe even in a distributed thingy without a central server, private rooms and much more is thinkable. You can checkout the README for more ideas.

So, that’s PathfinderBeacon in a nutshell—a simple, secure way to manage node addresses within distributed systems. It’s a small project with big potential, and I’m excited to see how it can support and simplify the management of distributed networks.

Thanks for reading, and I’m looking forward to constructive feedback or contributions!

You can find PathfinderBeacon on GitHub