The Challenge of Context Bloat in Ongoing Conversations
When using large language models (LLMs) for long-term system development or brainstorming over extended periods, you may encounter cases where response accuracy degrades due to excessive information accumulation within a single chat thread.
Throughout the development process, conversations begin with initial requirements definition, followed by code generation, error occurrences, and repeated trial-and-error exchanges aimed at resolving those errors. As these accumulate as “state” within a single thread, the context window ends up containing a mix of “currently valid, up-to-date specifications” and “past discarded code and discussions”.
As the number of input tokens increases and the proportion of noise rises, the LLM’s attention mechanism is believed to become more easily scattered. Consequently, phenomena such as the AI being dragged down by past incorrect premises or hallucinating to ignore constraints specified just moments before can occur.
As a practical operational approach to reset this degradation in accuracy and maintain high performance, the method of “periodically discarding old threads and starting fresh, clean threads” becomes a valid option.
Human Fatigue Associated with Manual “Past Log Summarization”
When starting a new thread, it is necessary to carry over the prerequisites and finalized specifications of the ongoing project. At this point, humans tend to engage in the task of reviewing past conversation logs, selecting important decisions, and summarizing them into a new prompt.
However, the task of “organizing and summarizing past information” consumes significant cognitive resources (brain memory) that engineers should ideally devote to system design or coding. The state in which humans manually update and summarize documents each time to maintain past context can be seen as creating inefficiency, despite supposedly leveraging AI to improve efficiency.
Meticulously organizing and managing vast amounts of conversation logs and notes that may never be revisited can ultimately become a burden later on. The very act of accumulating information directly leads to increased management costs.
Uncertainty of “Memory” Features Within LLMs
In recent years, some LLMs have been equipped with features such as “memory” or “long-term memory” that retain user information across sessions. Users can explicitly instruct the AI to record information, and using this feature might seem to automate context management.
However, the exact specifications of LLM memory systems are not publicly disclosed, and the details of how data is processed and retained internally remain unclear.
In actual operation, immediately after the AI responds with “Recorded” on the chat, if you delete that thread, you may observe phenomena where the information is not carried over to the next session (it is forgotten). This could be due to a time lag in background save processing, or the possibility that recording fails because the source disappears before processing completes.
As a result, to ensure that information is reliably committed to internal memory, cases arise where you are forced to engage in inefficient confirmation work, such as “explicitly instructing recording, leaving the thread untouched for a while, then testing on a different day in a separate thread whether it was correctly remembered, before deleting the past thread”.
In situations where you want to discard threads immediately and stay agile, continuing to bear this uncertainty caused by the black box nature of the system, along with the accompanying manual confirmation work, becomes a factor that creates new human fatigue.
Leveraging NotebookLM as External Storage
As an approach to avoid the uncertainty of internal memory and reduce human fatigue, a configuration that physically separates Gemini (equivalent to CPU/RAM), which performs thinking and code generation, from the platform (equivalent to storage) that retains context, can be considered effective. This is where Google’s NotebookLM serves as a reliable external storage for context management.
The specific data flow is as follows:
Past conversation logs with Gemini are saved as text sources in NotebookLM without any human editing or selection. This may include trial-and-error processes and error logs that could become noise. Rather than humans manually determining “what is important” and organizing it, first, everything is retained as reliable external data.
5. Practice: Seamless Meta-Management Through Direct Integration
There is no need for humans to manually summarize the lengthy sources dumped into NotebookLM, or to copy and paste them into a new chat. Instead, leverage the “direct integration” that is a strength of the Google ecosystem.
The specific operational steps are very simple:
-
Distillation of Sources: Within NotebookLM, instruct it to “extract the backbone of specifications decided in this discussion” and save the highly pure summary as a new source. Delete the old, noise-laden log sources to keep the inside of NotebookLM clean.
-
Direct Loading into Gemini: When resuming work in a new thread, directly specify the NotebookLM notebook from Gemini’s chat input field and add it as context.
The key point of this operation is that humans can completely abandon the labor of “how to organize and carry over past logs”. Humans do not even copy and paste; they simply instruct Gemini to “refer to” NotebookLM.
By directly connecting NotebookLM (vector database), which searches for information, with Gemini (logic engine), which develops thinking from it, context handover is achieved while suppressing human fatigue.
Trade-offs and Limitations in Practical Operation
While this architecture significantly reduces human fatigue associated with information organization, it comes with several clear trade-offs and technical constraints in practical operation. It is important to accept these and incorporate them into system design.
A. Integration Overhead (Time and Cost)
Each time a new thread is started, the “initialization effort” of having Gemini load and understand the NotebookLM background occurs every time. Token consumption from loading also occurs.
This is a cost intentionally paid to maintain a clean state. Therefore, rather than applying this to minor tasks that can be completed in minutes, it is realistic to limit the scope of application to situations where the cost-performance is justified, such as restoring context across days or weeks, or handing over heavy requirements definitions.
B. NotebookLM Source Limits and “Disposable” Notebooks
NotebookLM has an upper limit on the number of sources per notebook (currently around several hundred). If you continue dumping all logs for long-term large projects, the notebook itself will eventually reach its limit.
To avoid this constraint, it is necessary to view notebooks not as “permanent second brains” but as “disposable caches per project (or phase)”. An operation is required where, before approaching the limit, a final overall summary is output, the old notebook is discarded, and migration to a new notebook is performed.
C. Security and the Wall of Sensitive Information
Dumping conversation logs as-is into a cloud environment carries the risk of permanently storing sensitive information such as unreleased source code and API keys. There are cases where this is not suitable for enterprise development, so operational rules such as performing masking processing within the session before dumping need to be established.
Clean Operation of Context
Completed threads can be deleted without worrying about whether internal memory has been committed, maintaining clean operation at all times.
Even paying the initialization cost of having the background understood each time, the approach of not accumulating unnecessary logs on hand and reusing only necessary context through direct integration of Gemini and NotebookLM prevents long-term increases in human fatigue.
This method of abandoning the labor of judgment and organization, and even the effort of copy-pasting, and managing context through collaboration between AIs within the ecosystem, becomes the most rational and agile engineering choice when adopted with an understanding of system constraints.