Posts on Mia Heidenstedthttps://heidenstedt.org/posts/Recent content in Posts on Mia Heidenstedt HugoenFri, 17 Apr 2026 14:45:47 +0000AI-Assisted Cognition Endangers Human Developmenthttps://heidenstedt.org/posts/2026/ai-assisted-cognition-endangers-human-development/Wed, 15 Apr 2026 17:30:54 +0200https://heidenstedt.org/posts/2026/ai-assisted-cognition-endangers-human-development/ Best viewed on the original page, where extended functionality like the footnote helper is available.

Cognition with the help of AI is already a significant force in our world1, resulting in humanity-sized missed opportunities and risks. In this article, we will explore the risks of AI-assisted cognition and how to use these tools without falling into the trap of intellectual stagnation.

What is AI-Assisted Cognition

To understand what AI-assisted cognition is, we first need to understand what cognition is.

“Cognitions are mental processes that deal with knowledge. They encompass psychological activities that acquire, store, retrieve, transform, or apply information. Cognitions are a pervasive part of mental life, helping individuals understand and interact with the world.” Q: Wikipedia

Cognition can be assisted by external static information or external cognition.
For example, most people would put a book into the category of external static information and a discussion about a topic with another human, because humans think and process information themselves, into the external cognition category.

But where do discussions with AIs fit in? They are able to process information that can result in original solutions2, but they are still static and currently cannot learn3.

How AI decelerates the evolution of ideas, culture and knowledge

In early 2026, the USA prepared to invade Greenland and, therefore, the EU4. Only a few months prior to that it was completely unthinkable that the USA would even think about threatening an invasion of Greenland. As AI base models are stuck in the past, they do not easily accept these events as real and often label them as “hypothetical”, “fake news”, or “impossible”. This also affects new models like Gemini 3 Pro, GLM-5 or GPT-5.3-codex5.

As most new LLMs are just post-trained on a base model that is relatively old, even when post-trained on new events, they do not completely utilize this information in their cognition and are still skewed towards the static patterns of the base model’s hidden states6. They basically think something different from what they say.

So you might see the problem7 here already: If a lot of people use AIs to discuss, write, autocomplete, and brainstorm, but AI cognition does not reflect new events and cultural changes, like the change in the relationship between the USA and the EU, new geopolitical realities, and the EU population’s stance toward the USA, people will be skewed toward these old patterns and ideas. Cultural change has to build and maintain momentum indefinitely to persist against the static cognitive skew of AIs.

The Dynamic Dialectic Substrate

Human knowledge and ideas, and thus human development, are highly dependent on the Dynamic Dialectic Substrate8.
Understanding the Dynamic Dialectic Substrate will help to understand how AI-assisted cognition can endanger human development and how to use AI-assisted cognition without endangering human development.

The Dynamic Dialectic Substrate is the sum of all local and global dialectic9 processes and conclusions. It is the fundamental foundation upon which all of humanity is built, and the origin of all thoughts, concepts, ideas, and solutions that humans utilize.

The Dynamic Dialectic Substrate creates new concepts through a process of qualitative merging existing concepts, which can happen in a single person, a group of people, or even globally.

A hierarchical logic diagram demonstrating the Dialectic of the “Dynamic Dialectic Substrate” by visualizing how basic premises about fire, rain, and cold merge through intermediate steps to form the final deduction that a hut prevents pain.

The above image is a narrow slice of the Dialectic process present in the Dynamic Dialectic Substrate. You can see how concepts merge and evolve in higher and higher concepts. In this example the following dialectic process emerges:

Stage 1:

  • “Cold is Painful” and “Fire is Hot” result in “Fire removes Cold-Pain”
  • “Significant Water extinguishes Fire” and “Rain is falling Water” result in “Strong Rain extinguishes Fire”
  • “Rain is falling Water” and “Hut has a roof” result in “Hut shelters from Rain”

Stage 2:

  • “Fire removes Cold-Pain” and “Strong Rain extinguishes Fire” result in “Rain extinguishes Fire and therefore causes Cold-Pain”
  • “Strong Rain extinguishes Fire” and “Hut shelters from Rain” result in “Inside a Hut, Fire survives Rain”

Stage 3:

  • “Rain extinguishes Fire and therefore causes Cold-Pain” and “Inside a Hut, Fire survives Rain” result in “Hut protects Fire and therefore protects against Cold-Pain”

How AI endangers human development (Cognitive Inbreeding)

Because LLMs prefer or skew toward certain patterns and concepts (known as inductive bias), even after post-training, they reduce the cognitive range when used as a tool for cognition at the population level. This is especially true if only a few AI models are used, or if many AI models share just a few base models. This will lead to a loss of diversity of ideas, concepts, and solutions, which will slow down human development.

You might think of this as a world in which a significant portion of the population is speaking to the same five people to discuss problems, the world, relationships, and basically anything. It is hard to overstate how much influence these five people would have on humanity, even if they try their absolute best to be as neutral and open as possible. Humans who speak with these five people would still have their thinking massively shifted, and this becomes a significant problem at the population level.

It is entirely possible that we already have lost paths to great scientific discoveries or cultural shifts because of AI-skew or unnoticed refusal.

I tried to visualize this problem in the following image that shows how the range of higher level concepts is skewed into the direction the base model prefers:

A “Before and After” network diagram showing how AI-Assisted cognition introduces “AI-Skew” and “unnoticed refusal,” resulting in a significant loss of cognitive range and solution options compared to unassisted human cognition.

To come back to the example of the USA invading Greenland: It is obvious that humans using AI to brainstorm the geopolitical future of the EU, the USA, and Greenland will encounter patterns skewed toward the base model’s “worldview.” This bias might prevent many in the EU from even considering the possibility of moving away from foreign services or software. Such a shift could have massive consequences, especially since the EU relies heavily on USA services and software that could be turned off at any time. If this AI-skew affects even single individuals of specific groups such as politicians, CEOs, managers, or scientists the impact can be already be significant because of their decision-making power.

Human-AI-Cognition Hygiene: How to use AI assisted cognition without suffering AI-skew

Because base models are incredibly expensive to train and carry resilient biases, those without access to a GPU cluster must accept that these issues exist. To avoid problems like AI-skew and unnoticed refusal, they should instead focus on using specific strategies to mitigate them.

Speaking and discussing with other humans is obviously the most effective way to mitigate these problems. It might also be wise to mention that if you already have a good idea of a solution through AI-assisted cognition, you have to be careful not to nudge other humans in your direction. Try not to use questions or hints that will nudge other humans to a solution or thought that you had through AI-assisted cognition as long the other person is exploring a cognitive path you had not explored yet.

Regarding solutions that involve direct AI use, our range of options is quite limited, and as of now there is no solution that would completely or partially solve this problem on a population scale. Here are options that at least widen the range of concepts and ideas one can get out of LLMs while sadly not mitigating the main problem:

  • Use Search Engines to find relevant sources of information or let the AI search for you via Web Search and prevent it from giving you a solution or thought directly.
  • Use a variety of AIs with different base models
  • Explore different “AI personas” that simulate different perspectives and thinking styles like: “You are Einstein”, “You are on Drug X”, “You are a deranged but distinguished sea otter”

Research and further reading

Coda

Even though we have indications and even some evidence that AI-assisted cognition can endanger human development, the extent and depth are still unknown and unclear. More outcome-focused research is needed to understand the significance. Since we do not have a second humanity to A/B test all of that, there will always be a lot of uncertainty and speculation on this topic, since no one can isolate their cognition from the influence of population-level AI-assisted cognitive skew if they want to participate with other humans or their creations, which must already be influenced by AI-skew if it has any significant influence.

For me it is not entirely clear how we will recognize the effects of AI-skew and unnoticed refusal on a population level. We cannot know what innovations, discoveries, and cultural changes we are missing because of it. Although I am sure there will be figures that will extrapolate small indications into all-consuming dooming narratives, as I might do a little bit here for the sake of argument and attention to be compliant to our shared attention economy, it is probably, as everything, not that easy.

It is also not easy to imagine solutions for all of that, but I, for my part, will certainly try to exercise more “Cognition Hygiene”… Apart from this, it is much, much more fun for me to speak with humans about thoughts and ideas than with AIs.

I’ve seen slow awareness about this incredibly important topic that I hope to be able to speed up a bit with this article and by giving people a framework to understand and speak about it. If people have no words about something, it is hard to think and speak about it. It will be interesting to see how this topic evolves.

The topic of AI-skew and AI-assisted cognition is full of unknowns and it would be lovely to speak with people about it. I hope this article can be a starting point for that. If you want to share your thoughts, or are interested in a conversation about that, you can mail me at ai-skew@i5h.eu

  1. Use is expanding rapidly – especially weekly use – though not uniformly. Across countries, the proportion of people who say they have ever used any AI system rose from 40% (2024) to 61% (2025); weekly use nearly doubled from 18% to 34%.

    Reuters Institute

     ↩︎

  2. AIs are able to fully or partially solve Erdős math problems and can find new proofs to previously known full or partial solutions.

    See: Solutions to Erdős problems where AI tools played a primary role ↩︎

  3. AIs are able to “learn” in a very limited way, through their context what is not permanent.

    See: Is In-Context Learning Learning and
    Learning from context is harder than we thought ↩︎

  4. Greenland is part of the EU in a political sense as Denmark is part of the EU and Greenland is part of Denmark and all Greenlanders are EU citizens. Legally it is a OCT of the EU, not a member state. To communicate it directly: many people I know and live in the EU have interpreted these invasion plans as a direct invasion of the EU. In any case, this should not be a conversation about the invasion plans of Greenland. This is just a very great and obvious example of base model refusal and AI-skew. Please stay on the topic of AI-assisted cognition. ↩︎

  5. See this File in which I tried GPT-5.3-codex, Gemini 3 Pro and Claude 4.6 ↩︎

  6. Rather than promoting conceptual integration, fine-tuning may act as a form of rote injection, reinforcing isolated facts without building robust representations. Consequently, the success of fine-tuning appears to depend not only on the added data but also on how well the target concept is already embedded in the model’s pre-training knowledge.

    Hier et al., 2025

    As our results suggested, some internal mechanisms are mostly developed during pre-training and not significantly altered by post-training, such as factual knowledge storage and the truthfulness direction.

    These findings further support our conclusion: post-training generally preserves the internal representation of truthfulness.

    Jiang et al., 2025

     ↩︎

  7. I think we could describe this Problem on a very high level as chaining our “synchronic cognition” to a “diachronic cognition anchor”. But this is not the Problem i want to speak about, please keep reading. ↩︎

  8. In the research phase of this article, I came to the conclusion that existing system models are insufficient as they do not describe the process of how human knowledge, ideas and concepts evolve and how they are connected in a form that makes the idea of this work easily understandable. That is why I propose the “Dynamic Dialectic Substrate” to describe a model of cognition including the resulting dynamics and evolution. I hope this system model helps to understand this article. I choose the name “Dynamic Dialectic Substrate” because it symbolizes the obvious dialectic process, but other than the popular understanding of dialectic, it is, in my understanding, not static and rather dynamic, which I wanted to explicitly include in the name. Also, although a substrate is usually thought of as something passive, it is used here in a very active way. The idea was that humans (and apparently also AIs) are the actors and the Dynamic Dialectic Substrate is just the pool or medium out of which the actors draw their dialectics and, in doing so, changing the substrate itself. One could also say that the Dynamic Dialectic Substrate is just Pragmatism (C.S. Peirce’s logic of abduction) or Evolutionary Epistemology… if you have this perspective, please ask yourself if it is really REALLY the same and if the Dynamic Dialectic Substrate is not a much better representation of what needs to be grasped here. Again, I am not trying here to replace Hegel or Peirce, it is rather a macro view what happens in a population level. Hegel and Peirce’s models are compatible with the DDS as they describe different levels that the DDS does not contradict.

    I know that there are many theories of cognition like Conceptual blending, Thesis-Antithesis-Synthesis and also in some sense Memetics, but they all catch only parts of what we need here to understand the problem, like they only describe the mechanism of cognition or the transport mechanism of memes. The Hegelian Dialectic is too abstract, widely misunderstood, and bloated while vague at the same time. For example, the Hegelian Dialectic is often perceived as static and not dynamic, although Hegel would probably be very angry about that. It is by the way a common misconception that the Thesis-Antithesis-Synthesis model is from Hegel, it is actually from Heinrich Moritz Chalybäus. ↩︎

  9. Dialectic, also known as the dialectical method, refers originally to dialogue between people holding different points of view about a subject but wishing to arrive at the truth through reasoned argument. Dialectic resembles debate, but the concept excludes subjective elements such as emotional appeal and rhetoric; the object is more an eventual and commonly held truth than the ‘winning’ of an (often binary) competition.

    Wikipedia

     ↩︎
]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

Cognition with the help of AI is already a significant force in our world1, resulting in humanity-sized missed opportunities and risks. In this article, we will explore the risks of AI-assisted cognition and how to use these tools without falling into the trap of intellectual stagnation.

What is AI-Assisted Cognition

To understand what AI-assisted cognition is, we first need to understand what cognition is.

“Cognitions are mental processes that deal with knowledge. They encompass psychological activities that acquire, store, retrieve, transform, or apply information. Cognitions are a pervasive part of mental life, helping individuals understand and interact with the world.” Q: Wikipedia

Cognition can be assisted by external static information or external cognition.
For example, most people would put a book into the category of external static information and a discussion about a topic with another human, because humans think and process information themselves, into the external cognition category.

But where do discussions with AIs fit in? They are able to process information that can result in original solutions2, but they are still static and currently cannot learn3.

How AI decelerates the evolution of ideas, culture and knowledge

In early 2026, the USA prepared to invade Greenland and, therefore, the EU4. Only a few months prior to that it was completely unthinkable that the USA would even think about threatening an invasion of Greenland. As AI base models are stuck in the past, they do not easily accept these events as real and often label them as “hypothetical”, “fake news”, or “impossible”. This also affects new models like Gemini 3 Pro, GLM-5 or GPT-5.3-codex5.

As most new LLMs are just post-trained on a base model that is relatively old, even when post-trained on new events, they do not completely utilize this information in their cognition and are still skewed towards the static patterns of the base model’s hidden states6. They basically think something different from what they say.

So you might see the problem7 here already: If a lot of people use AIs to discuss, write, autocomplete, and brainstorm, but AI cognition does not reflect new events and cultural changes, like the change in the relationship between the USA and the EU, new geopolitical realities, and the EU population’s stance toward the USA, people will be skewed toward these old patterns and ideas. Cultural change has to build and maintain momentum indefinitely to persist against the static cognitive skew of AIs.

The Dynamic Dialectic Substrate

Human knowledge and ideas, and thus human development, are highly dependent on the Dynamic Dialectic Substrate8.
Understanding the Dynamic Dialectic Substrate will help to understand how AI-assisted cognition can endanger human development and how to use AI-assisted cognition without endangering human development.

The Dynamic Dialectic Substrate is the sum of all local and global dialectic9 processes and conclusions. It is the fundamental foundation upon which all of humanity is built, and the origin of all thoughts, concepts, ideas, and solutions that humans utilize.

The Dynamic Dialectic Substrate creates new concepts through a process of qualitative merging existing concepts, which can happen in a single person, a group of people, or even globally.

A hierarchical logic diagram demonstrating the Dialectic of the “Dynamic Dialectic Substrate” by visualizing how basic premises about fire, rain, and cold merge through intermediate steps to form the final deduction that a hut prevents pain.

The above image is a narrow slice of the Dialectic process present in the Dynamic Dialectic Substrate. You can see how concepts merge and evolve in higher and higher concepts. In this example the following dialectic process emerges:

Stage 1:

  • “Cold is Painful” and “Fire is Hot” result in “Fire removes Cold-Pain”
  • “Significant Water extinguishes Fire” and “Rain is falling Water” result in “Strong Rain extinguishes Fire”
  • “Rain is falling Water” and “Hut has a roof” result in “Hut shelters from Rain”

Stage 2:

  • “Fire removes Cold-Pain” and “Strong Rain extinguishes Fire” result in “Rain extinguishes Fire and therefore causes Cold-Pain”
  • “Strong Rain extinguishes Fire” and “Hut shelters from Rain” result in “Inside a Hut, Fire survives Rain”

Stage 3:

  • “Rain extinguishes Fire and therefore causes Cold-Pain” and “Inside a Hut, Fire survives Rain” result in “Hut protects Fire and therefore protects against Cold-Pain”

How AI endangers human development (Cognitive Inbreeding)

Because LLMs prefer or skew toward certain patterns and concepts (known as inductive bias), even after post-training, they reduce the cognitive range when used as a tool for cognition at the population level. This is especially true if only a few AI models are used, or if many AI models share just a few base models. This will lead to a loss of diversity of ideas, concepts, and solutions, which will slow down human development.

You might think of this as a world in which a significant portion of the population is speaking to the same five people to discuss problems, the world, relationships, and basically anything. It is hard to overstate how much influence these five people would have on humanity, even if they try their absolute best to be as neutral and open as possible. Humans who speak with these five people would still have their thinking massively shifted, and this becomes a significant problem at the population level.

It is entirely possible that we already have lost paths to great scientific discoveries or cultural shifts because of AI-skew or unnoticed refusal.

I tried to visualize this problem in the following image that shows how the range of higher level concepts is skewed into the direction the base model prefers:

A “Before and After” network diagram showing how AI-Assisted cognition introduces “AI-Skew” and “unnoticed refusal,” resulting in a significant loss of cognitive range and solution options compared to unassisted human cognition.

To come back to the example of the USA invading Greenland: It is obvious that humans using AI to brainstorm the geopolitical future of the EU, the USA, and Greenland will encounter patterns skewed toward the base model’s “worldview.” This bias might prevent many in the EU from even considering the possibility of moving away from foreign services or software. Such a shift could have massive consequences, especially since the EU relies heavily on USA services and software that could be turned off at any time. If this AI-skew affects even single individuals of specific groups such as politicians, CEOs, managers, or scientists the impact can be already be significant because of their decision-making power.

Human-AI-Cognition Hygiene: How to use AI assisted cognition without suffering AI-skew

Because base models are incredibly expensive to train and carry resilient biases, those without access to a GPU cluster must accept that these issues exist. To avoid problems like AI-skew and unnoticed refusal, they should instead focus on using specific strategies to mitigate them.

Speaking and discussing with other humans is obviously the most effective way to mitigate these problems. It might also be wise to mention that if you already have a good idea of a solution through AI-assisted cognition, you have to be careful not to nudge other humans in your direction. Try not to use questions or hints that will nudge other humans to a solution or thought that you had through AI-assisted cognition as long the other person is exploring a cognitive path you had not explored yet.

Regarding solutions that involve direct AI use, our range of options is quite limited, and as of now there is no solution that would completely or partially solve this problem on a population scale. Here are options that at least widen the range of concepts and ideas one can get out of LLMs while sadly not mitigating the main problem:

  • Use Search Engines to find relevant sources of information or let the AI search for you via Web Search and prevent it from giving you a solution or thought directly.
  • Use a variety of AIs with different base models
  • Explore different “AI personas” that simulate different perspectives and thinking styles like: “You are Einstein”, “You are on Drug X”, “You are a deranged but distinguished sea otter”

Research and further reading

Coda

Even though we have indications and even some evidence that AI-assisted cognition can endanger human development, the extent and depth are still unknown and unclear. More outcome-focused research is needed to understand the significance. Since we do not have a second humanity to A/B test all of that, there will always be a lot of uncertainty and speculation on this topic, since no one can isolate their cognition from the influence of population-level AI-assisted cognitive skew if they want to participate with other humans or their creations, which must already be influenced by AI-skew if it has any significant influence.

For me it is not entirely clear how we will recognize the effects of AI-skew and unnoticed refusal on a population level. We cannot know what innovations, discoveries, and cultural changes we are missing because of it. Although I am sure there will be figures that will extrapolate small indications into all-consuming dooming narratives, as I might do a little bit here for the sake of argument and attention to be compliant to our shared attention economy, it is probably, as everything, not that easy.

It is also not easy to imagine solutions for all of that, but I, for my part, will certainly try to exercise more “Cognition Hygiene”… Apart from this, it is much, much more fun for me to speak with humans about thoughts and ideas than with AIs.

I’ve seen slow awareness about this incredibly important topic that I hope to be able to speed up a bit with this article and by giving people a framework to understand and speak about it. If people have no words about something, it is hard to think and speak about it. It will be interesting to see how this topic evolves.

The topic of AI-skew and AI-assisted cognition is full of unknowns and it would be lovely to speak with people about it. I hope this article can be a starting point for that. If you want to share your thoughts, or are interested in a conversation about that, you can mail me at ai-skew@i5h.eu

  1. Use is expanding rapidly – especially weekly use – though not uniformly. Across countries, the proportion of people who say they have ever used any AI system rose from 40% (2024) to 61% (2025); weekly use nearly doubled from 18% to 34%.

    Reuters Institute

     ↩︎

  2. AIs are able to fully or partially solve Erdős math problems and can find new proofs to previously known full or partial solutions.

    See: Solutions to Erdős problems where AI tools played a primary role ↩︎

  3. AIs are able to “learn” in a very limited way, through their context what is not permanent.

    See: Is In-Context Learning Learning and
    Learning from context is harder than we thought ↩︎

  4. Greenland is part of the EU in a political sense as Denmark is part of the EU and Greenland is part of Denmark and all Greenlanders are EU citizens. Legally it is a OCT of the EU, not a member state. To communicate it directly: many people I know and live in the EU have interpreted these invasion plans as a direct invasion of the EU. In any case, this should not be a conversation about the invasion plans of Greenland. This is just a very great and obvious example of base model refusal and AI-skew. Please stay on the topic of AI-assisted cognition. ↩︎

  5. See this File in which I tried GPT-5.3-codex, Gemini 3 Pro and Claude 4.6 ↩︎

  6. Rather than promoting conceptual integration, fine-tuning may act as a form of rote injection, reinforcing isolated facts without building robust representations. Consequently, the success of fine-tuning appears to depend not only on the added data but also on how well the target concept is already embedded in the model’s pre-training knowledge.

    Hier et al., 2025

    As our results suggested, some internal mechanisms are mostly developed during pre-training and not significantly altered by post-training, such as factual knowledge storage and the truthfulness direction.

    These findings further support our conclusion: post-training generally preserves the internal representation of truthfulness.

    Jiang et al., 2025

     ↩︎

  7. I think we could describe this Problem on a very high level as chaining our “synchronic cognition” to a “diachronic cognition anchor”. But this is not the Problem i want to speak about, please keep reading. ↩︎

  8. In the research phase of this article, I came to the conclusion that existing system models are insufficient as they do not describe the process of how human knowledge, ideas and concepts evolve and how they are connected in a form that makes the idea of this work easily understandable. That is why I propose the “Dynamic Dialectic Substrate” to describe a model of cognition including the resulting dynamics and evolution. I hope this system model helps to understand this article. I choose the name “Dynamic Dialectic Substrate” because it symbolizes the obvious dialectic process, but other than the popular understanding of dialectic, it is, in my understanding, not static and rather dynamic, which I wanted to explicitly include in the name. Also, although a substrate is usually thought of as something passive, it is used here in a very active way. The idea was that humans (and apparently also AIs) are the actors and the Dynamic Dialectic Substrate is just the pool or medium out of which the actors draw their dialectics and, in doing so, changing the substrate itself. One could also say that the Dynamic Dialectic Substrate is just Pragmatism (C.S. Peirce’s logic of abduction) or Evolutionary Epistemology… if you have this perspective, please ask yourself if it is really REALLY the same and if the Dynamic Dialectic Substrate is not a much better representation of what needs to be grasped here. Again, I am not trying here to replace Hegel or Peirce, it is rather a macro view what happens in a population level. Hegel and Peirce’s models are compatible with the DDS as they describe different levels that the DDS does not contradict.

    I know that there are many theories of cognition like Conceptual blending, Thesis-Antithesis-Synthesis and also in some sense Memetics, but they all catch only parts of what we need here to understand the problem, like they only describe the mechanism of cognition or the transport mechanism of memes. The Hegelian Dialectic is too abstract, widely misunderstood, and bloated while vague at the same time. For example, the Hegelian Dialectic is often perceived as static and not dynamic, although Hegel would probably be very angry about that. It is by the way a common misconception that the Thesis-Antithesis-Synthesis model is from Hegel, it is actually from Heinrich Moritz Chalybäus. ↩︎

  9. Dialectic, also known as the dialectical method, refers originally to dialogue between people holding different points of view about a subject but wishing to arrive at the truth through reasoned argument. Dialectic resembles debate, but the concept excludes subjective elements such as emotional appeal and rhetoric; the object is more an eventual and commonly held truth than the ‘winning’ of an (often binary) competition.

    Wikipedia

     ↩︎
]]>How to effectively write quality code with AIhttps://heidenstedt.org/posts/2026/how-to-effectively-write-quality-code-with-ai/Fri, 06 Feb 2026 17:35:31 +0000https://heidenstedt.org/posts/2026/how-to-effectively-write-quality-code-with-ai/ Best viewed on the original page, where extended functionality like the footnote helper is available.

1 Establish a Clear Vision

You are a human, you know how this world behaves, how your team and colleagues behave, and what your users expect. You have experienced the world, and you want to work together with a system that has no experience in this world you live in. Every decision in your project that you don’t take and document will be taken for you by the AI.

Your responsibility of delivering quality code cannot be met if not even you know where long-lasting and difficult-to-change decisions are taken.
You must know what parts of your code need to be thought through and what must be vigorously tested.

Think about and discuss the architecture, interfaces, data structures, and algorithms you want to use. Think about how to test and validate your code to these specifications.

2 Maintain Precise documentation

You need to communicate to the AI in detail what you want to achieve, otherwise it will result in code that is unusable for your purpose.

Other developers also need to communicate this information to the AI. That makes it efficient to write as much documentation as practical in a standardized format and into the code repository itself.

Document the requirements, specifications, constraints, and architecture of your project in detail.
Document your coding standards, best practices, and design patterns.
Use flowcharts, UML diagrams, and other visual aids to communicate complex structures and workflows.
Write pseudocode for complex algorithms and logic to guide the AI in understanding your intentions.

3 Build debug systems that aid the AI

Develop efficient debug systems for the AI to use, reducing the need for multiple expensive CLI commands or browsers to verify code functionality. This will save time and resources while simplifying the process for the AI to identify and resolve code issues.

For example: Build a system that collects logs from all nodes in a distributed system and provides abstracted information like “The Data was send to all nodes”, “The Data X is saved on Node 1 but not on Node 2”.

4 Mark code review levels

Not all code is equally important. Some parts of your codebase are critical and need to be reviewed with extra care. Other parts are less important and can be generated with less oversight.

Use a system that allows you to mark how thoroughly each function has been reviewed.

For example you can use a prompt that will let the AI put the comment //A behind functions it wrote to indicate that the function has been written by an AI and is not yet reviewed by a human.

5 Write high level specifications and test by yourself

AIs will cheat and use shortcuts eventually. They will write mocks, stubs, and hard coded values to make the code tests succeed while the code itself is not working and most of the time dangerous. Often AIs will adapt or outright delete test code to let the code pass tests.

You must discourage this behavior by writing property based high level specification tests yourself. Build them in a way that makes it hard for the AI to cheat without having big code segments dedicated to it.
For example, use property based testing, restart the server and check in between if the database has the correct values.

Separate these test so the AI cannot edit them and prompt the AI not to change them.

6 Write interface tests in a separate context

Let an AI write property based interface tests for the expected behavior with as little context of the rest of the code as possible.
This will generate tests that are uninfluenced by the “implementation AI” which will prevent the tests from being adapted to the implementation in a way that makes them useless or less effective.

Separate these tests so the AI cannot edit them without approval and prompt the AI not to change them.

7 Use strict linting and formatting rules

Use strict linting and formatting rules to ensure code quality and consistency. This will help you and your AI to find issues early.

8 Use context specific coding agent prompts

Save time and money by utilizing path specific coding agent prompts like CLAUDE.md.

You can generate them automatically which will give your AI information it would otherwise as to create from scratch every time.

Try to provide as much high level information as practical, such as coding standards, best practices, design patterns, and specific requirements for the project. This will help the AI to generate code that is more aligned with your expectations and will reduce lookup time and cost.

9 Find and mark functions that have a high security risk

Identify and mark functions that have a high security risk, such as authentication, authorization, and data handling. These functions should be reviewed and tested with extra care and in such a way that a human has comprehended the logic of the function in all its dimensions and is confident about its correctness and safety.

Make this explicit with a comment like //HIGH-RISK-UNREVIEWED and //HIGH-RISK-REVIEWED to make sure that other developers are aware of the importance of these functions and will review them with extra care.

Make sure that the AI is instructed to change the review state of these functions as soon as it changes a single character in the function.
Developers must make sure that the status of these functions is always correct.

10 Reduce code complexity where possible

Aim to reduce the complexity of the generated code where possible. Each single line of code will eat up your context window and make it harder for the AI and You to keep track of the overall logic of your code.
Each avoidable line of code is costing energy, money and probability of future unsuccessful AI tasks.

11 Explore problems with experiments and prototypes

AI written code is cheap, use this to your advantage by exploring different solutions to a problem with experiments and prototypes with minimal specifications. This will allow you to find the best solution to a problem without investing too much time and resources in a single solution.

12 Do not generate blindly or to much complexity at once

Break down complex tasks into smaller, manageable tasks for the AI. Instead of asking the AI to generate the complete project or component at once, break it down into smaller tasks, such as generating individual functions or classes. This will help you to maintain control over the code and it’s logic.

You have to check each component or module for its adherence to the specifications and requirements.
If you have lost the overview of the complexity and inner workings of the code, you have lost control over your code and must restart from a state where you were in control of your code.

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

1 Establish a Clear Vision

You are a human, you know how this world behaves, how your team and colleagues behave, and what your users expect. You have experienced the world, and you want to work together with a system that has no experience in this world you live in. Every decision in your project that you don’t take and document will be taken for you by the AI.

Your responsibility of delivering quality code cannot be met if not even you know where long-lasting and difficult-to-change decisions are taken.
You must know what parts of your code need to be thought through and what must be vigorously tested.

Think about and discuss the architecture, interfaces, data structures, and algorithms you want to use. Think about how to test and validate your code to these specifications.

2 Maintain Precise documentation

You need to communicate to the AI in detail what you want to achieve, otherwise it will result in code that is unusable for your purpose.

Other developers also need to communicate this information to the AI. That makes it efficient to write as much documentation as practical in a standardized format and into the code repository itself.

Document the requirements, specifications, constraints, and architecture of your project in detail.
Document your coding standards, best practices, and design patterns.
Use flowcharts, UML diagrams, and other visual aids to communicate complex structures and workflows.
Write pseudocode for complex algorithms and logic to guide the AI in understanding your intentions.

3 Build debug systems that aid the AI

Develop efficient debug systems for the AI to use, reducing the need for multiple expensive CLI commands or browsers to verify code functionality. This will save time and resources while simplifying the process for the AI to identify and resolve code issues.

For example: Build a system that collects logs from all nodes in a distributed system and provides abstracted information like “The Data was send to all nodes”, “The Data X is saved on Node 1 but not on Node 2”.

4 Mark code review levels

Not all code is equally important. Some parts of your codebase are critical and need to be reviewed with extra care. Other parts are less important and can be generated with less oversight.

Use a system that allows you to mark how thoroughly each function has been reviewed.

For example you can use a prompt that will let the AI put the comment //A behind functions it wrote to indicate that the function has been written by an AI and is not yet reviewed by a human.

5 Write high level specifications and test by yourself

AIs will cheat and use shortcuts eventually. They will write mocks, stubs, and hard coded values to make the code tests succeed while the code itself is not working and most of the time dangerous. Often AIs will adapt or outright delete test code to let the code pass tests.

You must discourage this behavior by writing property based high level specification tests yourself. Build them in a way that makes it hard for the AI to cheat without having big code segments dedicated to it.
For example, use property based testing, restart the server and check in between if the database has the correct values.

Separate these test so the AI cannot edit them and prompt the AI not to change them.

6 Write interface tests in a separate context

Let an AI write property based interface tests for the expected behavior with as little context of the rest of the code as possible.
This will generate tests that are uninfluenced by the “implementation AI” which will prevent the tests from being adapted to the implementation in a way that makes them useless or less effective.

Separate these tests so the AI cannot edit them without approval and prompt the AI not to change them.

7 Use strict linting and formatting rules

Use strict linting and formatting rules to ensure code quality and consistency. This will help you and your AI to find issues early.

8 Use context specific coding agent prompts

Save time and money by utilizing path specific coding agent prompts like CLAUDE.md.

You can generate them automatically which will give your AI information it would otherwise as to create from scratch every time.

Try to provide as much high level information as practical, such as coding standards, best practices, design patterns, and specific requirements for the project. This will help the AI to generate code that is more aligned with your expectations and will reduce lookup time and cost.

9 Find and mark functions that have a high security risk

Identify and mark functions that have a high security risk, such as authentication, authorization, and data handling. These functions should be reviewed and tested with extra care and in such a way that a human has comprehended the logic of the function in all its dimensions and is confident about its correctness and safety.

Make this explicit with a comment like //HIGH-RISK-UNREVIEWED and //HIGH-RISK-REVIEWED to make sure that other developers are aware of the importance of these functions and will review them with extra care.

Make sure that the AI is instructed to change the review state of these functions as soon as it changes a single character in the function.
Developers must make sure that the status of these functions is always correct.

10 Reduce code complexity where possible

Aim to reduce the complexity of the generated code where possible. Each single line of code will eat up your context window and make it harder for the AI and You to keep track of the overall logic of your code.
Each avoidable line of code is costing energy, money and probability of future unsuccessful AI tasks.

11 Explore problems with experiments and prototypes

AI written code is cheap, use this to your advantage by exploring different solutions to a problem with experiments and prototypes with minimal specifications. This will allow you to find the best solution to a problem without investing too much time and resources in a single solution.

12 Do not generate blindly or to much complexity at once

Break down complex tasks into smaller, manageable tasks for the AI. Instead of asking the AI to generate the complete project or component at once, break it down into smaller tasks, such as generating individual functions or classes. This will help you to maintain control over the code and it’s logic.

You have to check each component or module for its adherence to the specifications and requirements.
If you have lost the overview of the complexity and inner workings of the code, you have lost control over your code and must restart from a state where you were in control of your code.

]]>Releasing: GoQueueBenchhttps://heidenstedt.org/posts/2025/releasing-goqueuebench/Tue, 25 Mar 2025 15:44:38 +0000https://heidenstedt.org/posts/2025/releasing-goqueuebench/ Best viewed on the original page, where extended functionality like the footnote helper is available.

As i coded on OuroborosDB i noticed that i need a very fast queue for a rather unique architectural design decision.
I try to build the network module in such a way that i can test the behavior completely deterministic while “simulating” entire clusters in a single process.

So i build a test prototype of my global network queue with go’s channels and noticed that it was a major performance bottleneck, after writing 2 different ring buffer queue implementations it became clear that some queues behave completely different under different congestion levels and core counts - some so unpredictable that i just did not wanted to use them in my project.

This prompted me to take a relatively large chunk out of my free time and write a suite to benchmark different queue implementations i build under different conditions and score them based on their performance and predictability.

The result of this work is GoQueueBench

These are the results of the benchmark suite:

Implementation Overall Score Throughput Light Load Throughput Heavy Load Throughput Average Stability Ratio Homogeneity Factor Uncertainty Total Tests
VortexQueue 11341466 6926449 5502925 8776309 1.15 0.87 0.25 681
LightningQueue 9631771 6638213 4627690 6036728 0.99 0.95 0.31 681
FastMPMCQueue 9384067 6870924 4598620 6070151 0.96 0.93 0.28 681
OptimizedMPMCQueue 9105262 6436385 4379823 5838555 0.97 0.94 0.32 681
OptimizedMPMCQueueSharded 8130197 6369891 3834140 6781865 0.84 0.88 0.39 681
MultiHeadQueue 7391203 4363332 3492068 5558849 1.12 0.91 0.36 681
BasicMPMCQueue 5599252 4370889 2669612 3667715 0.89 0.93 0.30 681
Golang Buffered Channel 5312485 6667828 2760985 4312720 0.54 0.82 0.66 681
FastMPMCQueueTicket 3229780 7705164 1203924 5803821 0.21 0.64 1.19 681

Please note that i build the package so that all queue adhere to the same interface and can be swapped out easily.

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

As i coded on OuroborosDB i noticed that i need a very fast queue for a rather unique architectural design decision.
I try to build the network module in such a way that i can test the behavior completely deterministic while “simulating” entire clusters in a single process.

So i build a test prototype of my global network queue with go’s channels and noticed that it was a major performance bottleneck, after writing 2 different ring buffer queue implementations it became clear that some queues behave completely different under different congestion levels and core counts - some so unpredictable that i just did not wanted to use them in my project.

This prompted me to take a relatively large chunk out of my free time and write a suite to benchmark different queue implementations i build under different conditions and score them based on their performance and predictability.

The result of this work is GoQueueBench

These are the results of the benchmark suite:

Implementation Overall Score Throughput Light Load Throughput Heavy Load Throughput Average Stability Ratio Homogeneity Factor Uncertainty Total Tests
VortexQueue 11341466 6926449 5502925 8776309 1.15 0.87 0.25 681
LightningQueue 9631771 6638213 4627690 6036728 0.99 0.95 0.31 681
FastMPMCQueue 9384067 6870924 4598620 6070151 0.96 0.93 0.28 681
OptimizedMPMCQueue 9105262 6436385 4379823 5838555 0.97 0.94 0.32 681
OptimizedMPMCQueueSharded 8130197 6369891 3834140 6781865 0.84 0.88 0.39 681
MultiHeadQueue 7391203 4363332 3492068 5558849 1.12 0.91 0.36 681
BasicMPMCQueue 5599252 4370889 2669612 3667715 0.89 0.93 0.30 681
Golang Buffered Channel 5312485 6667828 2760985 4312720 0.54 0.82 0.66 681
FastMPMCQueueTicket 3229780 7705164 1203924 5803821 0.21 0.64 1.19 681

Please note that i build the package so that all queue adhere to the same interface and can be swapped out easily.

]]>TTS on my Bloghttps://heidenstedt.org/posts/2025/tts-on-my-blog/Fri, 17 Jan 2025 16:06:00 +0200https://heidenstedt.org/posts/2025/tts-on-my-blog/ Best viewed on the original page, where extended functionality like the footnote helper is available.

For specific posts, I added an audio version to enhance accessability and maybe it is just nice to have something read it to you.
I am using Kokoro-TTS which is the first open weight TTS model that has sufficient quality for me to use it on my blog.

It is so nice to be finally able to use this technology, truly a dream come true.
Also i am looking into the possibility to use it for books that have no audio version yet. There is a good blogpost about how to convert an epub to an audiobook using this model and audiblez. But i have found that the emotional range and pauses of the result is not authentic enough and this kinda hurts understandability in situations where you are not focused on the audio. I hope future models will be able to bring a bit more life into the audio.

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

For specific posts, I added an audio version to enhance accessability and maybe it is just nice to have something read it to you.
I am using Kokoro-TTS which is the first open weight TTS model that has sufficient quality for me to use it on my blog.

It is so nice to be finally able to use this technology, truly a dream come true.
Also i am looking into the possibility to use it for books that have no audio version yet. There is a good blogpost about how to convert an epub to an audiobook using this model and audiblez. But i have found that the emotional range and pauses of the result is not authentic enough and this kinda hurts understandability in situations where you are not focused on the audio. I hope future models will be able to bring a bit more life into the audio.

]]>DETAILS part 3, a image serieshttps://heidenstedt.org/posts/2024/details-part-3-a-image-series/Wed, 20 Nov 2024 09:17:10 +0000https://heidenstedt.org/posts/2024/details-part-3-a-image-series/ Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a street photography series captured during my travels and excursions. I am considering selling prints of these images, but none currently exist. If you would like a print version of my work, please contact me and we can discuss the details.

Street photography featuring scenes across Italy.

Total images: 31
img/1.jpg
img/DSCF3874.jpg
img/DSCF3880-HDR.jpg
img/DSCF4047.jpg
img/DSCF4090.jpg
img/DSCF4110.jpg
img/DSCF4134.jpg
img/DSCF4234.jpg
img/DSCF4266.jpg
img/DSCF4269.jpg
img/DSCF4272.jpg
img/DSCF4275.jpg
img/DSCF4284.jpg
img/DSCF4323-2.jpg
img/DSCF4323.jpg
img/DSCF4335-HDR.jpg
img/DSCF4399.jpg
img/DSCF4419.jpg
img/DSCF4461.jpg
img/DSCF4473.jpg
img/DSCF4506.jpg
img/DSCF4542.jpg
img/DSCF4602.jpg
img/DSCF4630.jpg
img/DSCF4637.jpg
img/DSCF4647.jpg
img/DSCF4653.jpg
img/DSCF4655.jpg
img/DSCF4696-HDR.jpg
img/DSCF4793.jpg
img/DSCF4797.jpg
]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a street photography series captured during my travels and excursions. I am considering selling prints of these images, but none currently exist. If you would like a print version of my work, please contact me and we can discuss the details.

Street photography featuring scenes across Italy.

Total images: 31
img/1.jpg
img/DSCF3874.jpg
img/DSCF3880-HDR.jpg
img/DSCF4047.jpg
img/DSCF4090.jpg
img/DSCF4110.jpg
img/DSCF4134.jpg
img/DSCF4234.jpg
img/DSCF4266.jpg
img/DSCF4269.jpg
img/DSCF4272.jpg
img/DSCF4275.jpg
img/DSCF4284.jpg
img/DSCF4323-2.jpg
img/DSCF4323.jpg
img/DSCF4335-HDR.jpg
img/DSCF4399.jpg
img/DSCF4419.jpg
img/DSCF4461.jpg
img/DSCF4473.jpg
img/DSCF4506.jpg
img/DSCF4542.jpg
img/DSCF4602.jpg
img/DSCF4630.jpg
img/DSCF4637.jpg
img/DSCF4647.jpg
img/DSCF4653.jpg
img/DSCF4655.jpg
img/DSCF4696-HDR.jpg
img/DSCF4793.jpg
img/DSCF4797.jpg
]]>Release: OuroborosDB Data Storage Calculatorhttps://heidenstedt.org/posts/2024/release-ouroborosdb-data-storage-calculator/Fri, 01 Nov 2024 11:17:10 +0200https://heidenstedt.org/posts/2024/release-ouroborosdb-data-storage-calculator/ Best viewed on the original page, where extended functionality like the footnote helper is available.

I forgot to mention that i released a new tool for my project ouroboros-db, so here it is:

The “OuroborosDB Overhead Calculator” is a little tool which helps you to calculate the overhead of erasure coding systems while also considering the overhead of indexes and the blocks themselves. It is kinda fun to play around with so give it a try!

https://i5heu.github.io/ouroboros-db-overhead-calculator/

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

I forgot to mention that i released a new tool for my project ouroboros-db, so here it is:

The “OuroborosDB Overhead Calculator” is a little tool which helps you to calculate the overhead of erasure coding systems while also considering the overhead of indexes and the blocks themselves. It is kinda fun to play around with so give it a try!

https://i5heu.github.io/ouroboros-db-overhead-calculator/

]]>Ouroboros DB Dev Journal: Erasure Codinghttps://heidenstedt.org/posts/2024/ouroboros-db-dev-journal-erasure-coding/Tue, 27 Aug 2024 17:55:05 +0200https://heidenstedt.org/posts/2024/ouroboros-db-dev-journal-erasure-coding/ Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a Dev Journal for the Ouroboros DB Project.
I try to write down my thoughts and ideas Somewhat structured to have it as a reference for later and i publish it to give this information a chance to help other and to get feedback from the community.
If you have feedback you can write me on my Mastodon.

Pls note that this is only a Dev Journal and not a Blog Post, so it may be a bit unstructured and not as polished as a Blog Post, including typos and other errors.

TL;DR:

Today i worked on a refactoring of the architecture and feasibility of erasure coding for Ouroboros DB. There are some concerns i had regarding the potential index size and overhead that will result from it, and as it turns out, it is not as bad as i thought but i need a DHT for the index.

Architecture

I think it would be super handy to be able to run parts of data pipelines (eg. storing files into chunked,compressed, encrypted and then erasure coded blocks) in a distributed manner. for safety and because i want to be able to run trustless nodes that can’t see the raw data, it would be necessary to have the file chunking, compression and encryption part on the server the client currently speaks too, although it would be possible to pre-chunk the file on the client and upload it to different nodes, although here is the question if it is easy to port the chunking algorithm to browser JS or WASM.

Modules

Refactoring of the code is needed to implement erasure coding, maybe we can even get rid of the chunk entirely as a stored thing since we have the data in the parity blocks already.

Maybe best to add the erasure coding to the StoreDataPipeline and add the needed erasure coding metadata to ChunkData.

ASCII Art of the new architecture (click to get the .txt file):

ASCII Art of the new architecture

Erasure Coding

For now i think it is okay yo make the erasure coding a simple n6 k3, which would result in a parity block size of 327,680 Bytes or 0.31MB as average. This would result in about 1.875MB per chunk that is 1.25MB big in average. This would result in a 50% overhead for the erasure coding, which i think is quite okay.

For the goal of storing 100TB in it, which are about 400M Chunks, we would need 2,400,000,000 parity blocks, aka 2,4 Billion.

If we consider following overhead for the erasure coding:

Parity Meta Data This is the Meta Data for a `parity block`: 
{
"parityHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
"chunkHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
"sizeByte": 433000,
"lastChecked": 1724760126282,
"reblanceLog": [
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    }
],
"userLog": [
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    }
],
"this is for the KV Key": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0"
}

We see that parity block metadata has under 2373 Bytes of overhead, which is about 0.72%% of the parity block size (very good). If one node would need to store the entire hash table we look at a size of 5.18TB, which means that we need a DHT for this to work. This also is the case for the Events, which introduce more storage overhead (more in DHT )

DHT

We need additional DHT Metadata like this:

DHT Meta Data 
{
  "parityHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
  "storingNodes": [
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    }
  ]
}

This example meta data has a size of 674 Bytes, which is about 0.21% of the parity block size. The DHT meta data would require an additional 1.47TB of storage.

Overhead

DHT meta data in combination with the parity block meta data would have a total overhead of 0.93%% of the erasure coding metadata relative to the erasure coding parity block size, the entire metadata overhead at 100TB and 2,4 Billion parity blocks would be 6.65TB.

Adding this to the overhead from the erasure coding which are 50% we would have a total overhead of 56.65% relative to the raw data size. If we have a utilization of each chunk of 1310720 Bits which has been achieved with 10MB files with random binary data. This would result in a HDD to raw data ratio of 43.35% (which is pretty good for a k6n3 config and having self healing capabilities) for following configuration:

  • 100TB of raw data
  • data deduplication via BuzzHash into chunks
  • chunk compression via Zstd
  • chunk encryption via AES256
  • each chunk becomes 6 parity blocks of which 3 can be lost without data loss

Conclusion

I have written a Google Calc Spreadsheet for it, which you can find here: ouroboros-db Overhead Calculator.

Though this Spreadsheet is neat, i discoverd that i need some surface plotting to find the best configuration for the erasure coding.
Sadly there is no way to do this in Google Calc, so i will need to write a small Browser App for it… maybe i use Svelte for it, i haven’t used it in a while now.

Further Reading

A good overview over what erasure coding is about: Erasure Coding for Distributed Systems
Also See HN comments

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a Dev Journal for the Ouroboros DB Project.
I try to write down my thoughts and ideas Somewhat structured to have it as a reference for later and i publish it to give this information a chance to help other and to get feedback from the community.
If you have feedback you can write me on my Mastodon.

Pls note that this is only a Dev Journal and not a Blog Post, so it may be a bit unstructured and not as polished as a Blog Post, including typos and other errors.

TL;DR:

Today i worked on a refactoring of the architecture and feasibility of erasure coding for Ouroboros DB. There are some concerns i had regarding the potential index size and overhead that will result from it, and as it turns out, it is not as bad as i thought but i need a DHT for the index.

Architecture

I think it would be super handy to be able to run parts of data pipelines (eg. storing files into chunked,compressed, encrypted and then erasure coded blocks) in a distributed manner. for safety and because i want to be able to run trustless nodes that can’t see the raw data, it would be necessary to have the file chunking, compression and encryption part on the server the client currently speaks too, although it would be possible to pre-chunk the file on the client and upload it to different nodes, although here is the question if it is easy to port the chunking algorithm to browser JS or WASM.

Modules

Refactoring of the code is needed to implement erasure coding, maybe we can even get rid of the chunk entirely as a stored thing since we have the data in the parity blocks already.

Maybe best to add the erasure coding to the StoreDataPipeline and add the needed erasure coding metadata to ChunkData.

ASCII Art of the new architecture (click to get the .txt file):

ASCII Art of the new architecture

Erasure Coding

For now i think it is okay yo make the erasure coding a simple n6 k3, which would result in a parity block size of 327,680 Bytes or 0.31MB as average. This would result in about 1.875MB per chunk that is 1.25MB big in average. This would result in a 50% overhead for the erasure coding, which i think is quite okay.

For the goal of storing 100TB in it, which are about 400M Chunks, we would need 2,400,000,000 parity blocks, aka 2,4 Billion.

If we consider following overhead for the erasure coding:

Parity Meta Data This is the Meta Data for a `parity block`: 
{
"parityHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
"chunkHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
"sizeByte": 433000,
"lastChecked": 1724760126282,
"reblanceLog": [
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    }
],
"userLog": [
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    },
    {
    "time": 1724760126282,
    "from": "2deb000b57bfac9d72c14d4ed967b572"
    }
],
"this is for the KV Key": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0"
}

We see that parity block metadata has under 2373 Bytes of overhead, which is about 0.72%% of the parity block size (very good). If one node would need to store the entire hash table we look at a size of 5.18TB, which means that we need a DHT for this to work. This also is the case for the Events, which introduce more storage overhead (more in DHT )

DHT

We need additional DHT Metadata like this:

DHT Meta Data 
{
  "parityHash": "aeae379a6e857728e44164267fdb7a0e27b205d757cc19899586c89dbb221930f1813d02ff93a661859bc17065eac4d6edf3c38a034e6283a84754d52917e5b0",
  "storingNodes": [
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    },
    {
      "node": "2deb000b57bfac9d72c14d4ed967b572",
      "lastValidated": 1724760126282
    }
  ]
}

This example meta data has a size of 674 Bytes, which is about 0.21% of the parity block size. The DHT meta data would require an additional 1.47TB of storage.

Overhead

DHT meta data in combination with the parity block meta data would have a total overhead of 0.93%% of the erasure coding metadata relative to the erasure coding parity block size, the entire metadata overhead at 100TB and 2,4 Billion parity blocks would be 6.65TB.

Adding this to the overhead from the erasure coding which are 50% we would have a total overhead of 56.65% relative to the raw data size. If we have a utilization of each chunk of 1310720 Bits which has been achieved with 10MB files with random binary data. This would result in a HDD to raw data ratio of 43.35% (which is pretty good for a k6n3 config and having self healing capabilities) for following configuration:

  • 100TB of raw data
  • data deduplication via BuzzHash into chunks
  • chunk compression via Zstd
  • chunk encryption via AES256
  • each chunk becomes 6 parity blocks of which 3 can be lost without data loss

Conclusion

I have written a Google Calc Spreadsheet for it, which you can find here: ouroboros-db Overhead Calculator.

Though this Spreadsheet is neat, i discoverd that i need some surface plotting to find the best configuration for the erasure coding.
Sadly there is no way to do this in Google Calc, so i will need to write a small Browser App for it… maybe i use Svelte for it, i haven’t used it in a while now.

Further Reading

A good overview over what erasure coding is about: Erasure Coding for Distributed Systems
Also See HN comments

]]>My New Public Service: PathfinderBeaconhttps://heidenstedt.org/posts/2024/my-new-public-service-pathfinderbeacon/Fri, 23 Aug 2024 16:00:00 +0200https://heidenstedt.org/posts/2024/my-new-public-service-pathfinderbeacon/ Best viewed on the original page, where extended functionality like the footnote helper is available.

In the midst of working on my project, ouroboros-db, I realized I needed a straightforward way to locate other nodes using the same RSA key. That’s when PathfinderBeacon came into being and that i release to the public with these words.

What is PathfinderBeacon?

PathfinderBeacon is a tool designed to make it easier to bootstrap your distributed systems using just a RSA key and one of the most robust dictionaries available: DNS. Since DNS inherently caches information at various levels, it allows PathfinderBeacon to deliver node addresses quickly and with minimal bandwidth usage. This means if multiple searches are conducted within the same network for the same room, there’s a high chance the requests won’t even leave your local network, thanks to the caching capabilities of your router’s local DNS.

You can think of it like a DDNS (Dynamic Domain Name System) service, but you only need an RSA key to write to a subdomain. The subdomain is generated from the RSA key, and there is little surface for abuse since you can only publish highly sanitized TXT records.

How It Works

Setting up is pretty simple. You either provide an RSA key or let PathfinderBeacon generate one for you. Once set, you can publish your room and node addresses, and anyone with the same RSA key can write to the same room, while anyone can read from it. This makes it easy to bootstrap your distributed systems without the need of running your own bootstrap service.

Currently, there is an official Golang library for interacting with PathfinderBeacon, which you can find here. If you’re interested in developing libraries for other programming languages, feel free to reach out, and I’d be glad to assist and subsequently feature your work on the project repository.

Future Directions

For me this project is pretty much finished since it solves the underlying problem quite effectively, but i am fascinated by this problem and i would love to develop PathfinderBeacon further, maybe even in a distributed thingy without a central server, private rooms and much more is thinkable. You can checkout the README for more ideas.

So, that’s PathfinderBeacon in a nutshell—a simple, secure way to manage node addresses within distributed systems. It’s a small project with big potential, and I’m excited to see how it can support and simplify the management of distributed networks.

Thanks for reading, and I’m looking forward to constructive feedback or contributions!

You can find PathfinderBeacon on GitHub

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

In the midst of working on my project, ouroboros-db, I realized I needed a straightforward way to locate other nodes using the same RSA key. That’s when PathfinderBeacon came into being and that i release to the public with these words.

What is PathfinderBeacon?

PathfinderBeacon is a tool designed to make it easier to bootstrap your distributed systems using just a RSA key and one of the most robust dictionaries available: DNS. Since DNS inherently caches information at various levels, it allows PathfinderBeacon to deliver node addresses quickly and with minimal bandwidth usage. This means if multiple searches are conducted within the same network for the same room, there’s a high chance the requests won’t even leave your local network, thanks to the caching capabilities of your router’s local DNS.

You can think of it like a DDNS (Dynamic Domain Name System) service, but you only need an RSA key to write to a subdomain. The subdomain is generated from the RSA key, and there is little surface for abuse since you can only publish highly sanitized TXT records.

How It Works

Setting up is pretty simple. You either provide an RSA key or let PathfinderBeacon generate one for you. Once set, you can publish your room and node addresses, and anyone with the same RSA key can write to the same room, while anyone can read from it. This makes it easy to bootstrap your distributed systems without the need of running your own bootstrap service.

Currently, there is an official Golang library for interacting with PathfinderBeacon, which you can find here. If you’re interested in developing libraries for other programming languages, feel free to reach out, and I’d be glad to assist and subsequently feature your work on the project repository.

Future Directions

For me this project is pretty much finished since it solves the underlying problem quite effectively, but i am fascinated by this problem and i would love to develop PathfinderBeacon further, maybe even in a distributed thingy without a central server, private rooms and much more is thinkable. You can checkout the README for more ideas.

So, that’s PathfinderBeacon in a nutshell—a simple, secure way to manage node addresses within distributed systems. It’s a small project with big potential, and I’m excited to see how it can support and simplify the management of distributed networks.

Thanks for reading, and I’m looking forward to constructive feedback or contributions!

You can find PathfinderBeacon on GitHub

]]>DETAILS part 2, a image serieshttps://heidenstedt.org/posts/2024/details-part-2-a-image-series/Thu, 04 Apr 2024 09:17:10 +0000https://heidenstedt.org/posts/2024/details-part-2-a-image-series/ Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a street photography series captured during my travels and excursions. I am considering selling prints of these images, but none currently exist. If you would like a print version of my work, please contact me and we can discuss the details.

Street photography featuring scenes across Berlin.

Total images: 9
img/1.png
img/2.png
img/3.png
img/4.png
img/5.png
img/6.png
img/7.png
img/8.png
img/9.png
]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a street photography series captured during my travels and excursions. I am considering selling prints of these images, but none currently exist. If you would like a print version of my work, please contact me and we can discuss the details.

Street photography featuring scenes across Berlin.

Total images: 9
img/1.png
img/2.png
img/3.png
img/4.png
img/5.png
img/6.png
img/7.png
img/8.png
img/9.png
]]>DETAILS part 1, a image serieshttps://heidenstedt.org/posts/2024/details-part-1-a-image-series/Mon, 05 Feb 2024 11:17:10 +0000https://heidenstedt.org/posts/2024/details-part-1-a-image-series/ Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a street photography series captured during my travels and excursions. I am considering selling prints of these images, but none currently exist. If you would like a print version of my work, please contact me and we can discuss the details.

Street photography featuring scenes across Switzerland.

Total images: 48
img/1.jpeg
img/10.jpeg
img/11.jpeg
img/12.jpeg
img/13.jpeg
img/14.jpeg
img/15.jpeg
img/16.jpeg
img/17.jpeg
img/18.jpeg
img/19.jpeg
img/2.jpeg
img/20.jpeg
img/21.jpeg
img/22.jpeg
img/23.jpeg
img/24.jpeg
img/25.jpeg
img/26.jpeg
img/27.jpeg
img/28.jpeg
img/29.jpeg
img/30.jpeg
img/31.jpeg
img/32.jpeg
img/33.jpeg
img/34.jpeg
img/35.jpeg
img/36.jpeg
img/38.jpeg
img/39.jpeg
img/4.jpeg
img/40.jpeg
img/41.jpeg
img/42.jpeg
img/43.jpeg
img/44.jpeg
img/45.jpeg
img/46.jpeg
img/47.jpeg
img/48.jpeg
img/49.jpeg
img/5.jpeg
img/50.jpeg
img/6.jpeg
img/7.jpeg
img/8.jpeg
img/9.jpeg
]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

This is a street photography series captured during my travels and excursions. I am considering selling prints of these images, but none currently exist. If you would like a print version of my work, please contact me and we can discuss the details.

Street photography featuring scenes across Switzerland.

Total images: 48
img/1.jpeg
img/10.jpeg
img/11.jpeg
img/12.jpeg
img/13.jpeg
img/14.jpeg
img/15.jpeg
img/16.jpeg
img/17.jpeg
img/18.jpeg
img/19.jpeg
img/2.jpeg
img/20.jpeg
img/21.jpeg
img/22.jpeg
img/23.jpeg
img/24.jpeg
img/25.jpeg
img/26.jpeg
img/27.jpeg
img/28.jpeg
img/29.jpeg
img/30.jpeg
img/31.jpeg
img/32.jpeg
img/33.jpeg
img/34.jpeg
img/35.jpeg
img/36.jpeg
img/38.jpeg
img/39.jpeg
img/4.jpeg
img/40.jpeg
img/41.jpeg
img/42.jpeg
img/43.jpeg
img/44.jpeg
img/45.jpeg
img/46.jpeg
img/47.jpeg
img/48.jpeg
img/49.jpeg
img/5.jpeg
img/50.jpeg
img/6.jpeg
img/7.jpeg
img/8.jpeg
img/9.jpeg
]]>Blog Infrastructure Update 2https://heidenstedt.org/posts/2022/blog-infrastructure-update-2/Wed, 03 Aug 2022 21:17:10 +0200https://heidenstedt.org/posts/2022/blog-infrastructure-update-2/ Best viewed on the original page, where extended functionality like the footnote helper is available.

Sadly i did’t saved measurements form before the migration to the new Infrastructure. I will do better next time ;)

My blog has experienced a major infrastructure update.
Previously this blog was served by a tiny but highly optimized server at a Hetzner datacenter.
Since this blog is static it was for example possible to pre-compress the files, which shaved a massive 1-0.5ms from a cold TTBF.

Sadly this approach was a bit flawed because outside of germany the cold TTBF times weren’t exactly good.
Surprisingly CloudFlare did nothing to these times, since the cache of CloudFlare is short lived and not prefetched.

My solution to this problem, to make a global low cold TTBF possible, was BunnyCDN which provides a Service that can store and server static file, these files are replicated Globally to user-selectable locations.
At my case (i use all locations), the price per GB per month is 0.28$ which is very good if you consider that all content is replicated and served from SSDs in 14 locations.

A graph showing 14 locations of the SSDs

Here you can see where these locations are, green is the “main storage zone”, which is not configurable.

The switch to BunnyCDN gave me a immense performance boost around the world, and surprisingly also in germany.
BunnyCDN dose not allow for rsync or something usable to transfer the Files to them.

I use lftp to transfer the files via FTP, which takes about 9 min each time. This is tbh shitty, and i don’t like it a bit.

DNS and Peering

Sadly BunnyCDN dose not allow to use them as a nameServer, also i have to use a c-name to hook up to their CDN.
Cloudflare on the other side acts as a nameservers and returns direct IPv4s and IPv6s on the DNS-query, this will save 1 DNS-Request and is therefore faster, especially since most users using the DNS-Server from their ISP which are often very slow.

So i tried to put CloudFlare in front of BunnyCDN, which had a negative impact on the TTFB, although not a great one.
In same cases it is faster to get a uncached page on BunnyCDN through CloudFlare because CloudFlare has a lot and very good peering to ISPs and exchanges.

Sadly CloudFlare TTBF behaves a unpredictable.
It can be that you get in one request 20ms TTFB and the next one takes 300ms.

I think i will run CloudFlare anyway in front of BunnyCDN in hope to better mitigate Attacks and big page visit numbers.
I hope that CloudFlare will cache such surges for me… they can become expensive.

Costs

This entire setup cost me less then 1$ per Month (without the Domain), which is very very cheap for something this fast.
I think it is amazing how cheap and easy you can distribute your ideas and thoughts today, without relining on a platform that appropriates your content.

Conclusion

Costs less then 1$ per Month.
Scales amazingly.
Very fast global latency.
I don’t have to keep a server updated.

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

Sadly i did’t saved measurements form before the migration to the new Infrastructure. I will do better next time ;)

My blog has experienced a major infrastructure update.
Previously this blog was served by a tiny but highly optimized server at a Hetzner datacenter.
Since this blog is static it was for example possible to pre-compress the files, which shaved a massive 1-0.5ms from a cold TTBF.

Sadly this approach was a bit flawed because outside of germany the cold TTBF times weren’t exactly good.
Surprisingly CloudFlare did nothing to these times, since the cache of CloudFlare is short lived and not prefetched.

My solution to this problem, to make a global low cold TTBF possible, was BunnyCDN which provides a Service that can store and server static file, these files are replicated Globally to user-selectable locations.
At my case (i use all locations), the price per GB per month is 0.28$ which is very good if you consider that all content is replicated and served from SSDs in 14 locations.

A graph showing 14 locations of the SSDs

Here you can see where these locations are, green is the “main storage zone”, which is not configurable.

The switch to BunnyCDN gave me a immense performance boost around the world, and surprisingly also in germany.
BunnyCDN dose not allow for rsync or something usable to transfer the Files to them.

I use lftp to transfer the files via FTP, which takes about 9 min each time. This is tbh shitty, and i don’t like it a bit.

DNS and Peering

Sadly BunnyCDN dose not allow to use them as a nameServer, also i have to use a c-name to hook up to their CDN.
Cloudflare on the other side acts as a nameservers and returns direct IPv4s and IPv6s on the DNS-query, this will save 1 DNS-Request and is therefore faster, especially since most users using the DNS-Server from their ISP which are often very slow.

So i tried to put CloudFlare in front of BunnyCDN, which had a negative impact on the TTFB, although not a great one.
In same cases it is faster to get a uncached page on BunnyCDN through CloudFlare because CloudFlare has a lot and very good peering to ISPs and exchanges.

Sadly CloudFlare TTBF behaves a unpredictable.
It can be that you get in one request 20ms TTFB and the next one takes 300ms.

I think i will run CloudFlare anyway in front of BunnyCDN in hope to better mitigate Attacks and big page visit numbers.
I hope that CloudFlare will cache such surges for me… they can become expensive.

Costs

This entire setup cost me less then 1$ per Month (without the Domain), which is very very cheap for something this fast.
I think it is amazing how cheap and easy you can distribute your ideas and thoughts today, without relining on a platform that appropriates your content.

Conclusion

Costs less then 1$ per Month.
Scales amazingly.
Very fast global latency.
I don’t have to keep a server updated.

]]>Poverty and Financial Goals with Inflationhttps://heidenstedt.org/posts/2022/poverty-and-financial-goals-with-inflation/Wed, 08 Jun 2022 11:17:10 +0000https://heidenstedt.org/posts/2022/poverty-and-financial-goals-with-inflation/ Best viewed on the original page, where extended functionality like the footnote helper is available.

A while ago, I decided to work towards certain financial goals in order to have a buffer for bad times. Through some friends and acquaintances, I’ve seen very clearly how difficult it is to get back on your feet from a financially challenging situation. Being poor is incredibly expensive. Here, the relative impact on the basis for existence is particularly visible. If I overlook a bill and have to pay a 5€ reminder fee, I don’t even notice it in my daily life… not even a Starbucks coffee… In a difficult financial situation, this can mean a day without food.

This extends to dimensions such as “I can’t get an account with a bank and have to find an employer who can pay me with a physical wage packet,” to “I am forced to go to jail because my wage garnishment takes so much money that I can no longer afford the DB (Deutsche Bahn) ticket and I can’t possibly pay the court fine for ‘fare evasion’ in addition.”

Being poor is very, very expensive and incredibly unfair, and you are subject to massive arbitrariness.

One of my biggest fears is to get into such a practically hopeless situation, from which there is very rarely an escape.

So I’ve decided to save a certain amount of money that can keep me afloat for a while and afford investments like “I need a car for this job because the job is in the countryside.”

I thought that about 50,000€ should suffice to survive pretty much any financial blow, at least in the short and medium term.

Note: This is purely an arbitrary estimate of mine, a gut feeling. I think this value is very dependent on the risk you carry in life. So, for example, “Do I absolutely need a car”, “Is my house well insured”, “Do I have a child”, “How many long-term contracts do I have?”, etc. These are all risks that need to be cushioned in the event of job loss or the loss of work capacity. You should plan at least enough money so that you are able to reduce your expenses to fit into basic security, and that can take quite a long time, as with expensive mobile phone contracts, for example.

Inflation

Now we have the problem that the counter value of, for example, 50k€ decreases with inflation. Since we don’t just set a number and that’s enough, we’re essentially saving for performance that is available to us in case of need, we want to include inflation in these 50k€. After some searching, I came across the “HICP(2005-06)” index, which indicates the value development of the euro in relation to 06.2005. Today, on June 8, 2022, the last determined HICP is at 136.8. So we have to provide 136.8% of the money value of 2005 to get the same performance. To make the calculation faster and to have a simple, good source for the HICP, I built a small tool and pre-calculated some values: https://i5heu.github.io/HICP-calculator/index.html

50k€ in 2005 are today 68,400€.

]]> Best viewed on the original page, where extended functionality like the footnote helper is available.

A while ago, I decided to work towards certain financial goals in order to have a buffer for bad times. Through some friends and acquaintances, I’ve seen very clearly how difficult it is to get back on your feet from a financially challenging situation. Being poor is incredibly expensive. Here, the relative impact on the basis for existence is particularly visible. If I overlook a bill and have to pay a 5€ reminder fee, I don’t even notice it in my daily life… not even a Starbucks coffee… In a difficult financial situation, this can mean a day without food.

This extends to dimensions such as “I can’t get an account with a bank and have to find an employer who can pay me with a physical wage packet,” to “I am forced to go to jail because my wage garnishment takes so much money that I can no longer afford the DB (Deutsche Bahn) ticket and I can’t possibly pay the court fine for ‘fare evasion’ in addition.”

Being poor is very, very expensive and incredibly unfair, and you are subject to massive arbitrariness.

One of my biggest fears is to get into such a practically hopeless situation, from which there is very rarely an escape.

So I’ve decided to save a certain amount of money that can keep me afloat for a while and afford investments like “I need a car for this job because the job is in the countryside.”

I thought that about 50,000€ should suffice to survive pretty much any financial blow, at least in the short and medium term.

Note: This is purely an arbitrary estimate of mine, a gut feeling. I think this value is very dependent on the risk you carry in life. So, for example, “Do I absolutely need a car”, “Is my house well insured”, “Do I have a child”, “How many long-term contracts do I have?”, etc. These are all risks that need to be cushioned in the event of job loss or the loss of work capacity. You should plan at least enough money so that you are able to reduce your expenses to fit into basic security, and that can take quite a long time, as with expensive mobile phone contracts, for example.

Inflation

Now we have the problem that the counter value of, for example, 50k€ decreases with inflation. Since we don’t just set a number and that’s enough, we’re essentially saving for performance that is available to us in case of need, we want to include inflation in these 50k€. After some searching, I came across the “HICP(2005-06)” index, which indicates the value development of the euro in relation to 06.2005. Today, on June 8, 2022, the last determined HICP is at 136.8. So we have to provide 136.8% of the money value of 2005 to get the same performance. To make the calculation faster and to have a simple, good source for the HICP, I built a small tool and pre-calculated some values: https://i5heu.github.io/HICP-calculator/index.html

50k€ in 2005 are today 68,400€.

]]>