<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>laxmena</title>
    <link>https://laxmena.com/</link>
    <description></description>
    <pubDate>Mon, 06 Apr 2026 23:36:02 +0000</pubDate>
    <image>
      <url>https://i.snap.as/n9575tJN.png</url>
      <title>laxmena</title>
      <link>https://laxmena.com/</link>
    </image>
    <item>
      <title>Most people can&#39;t multitask. Stop pretending you can.</title>
      <link>https://laxmena.com/most-people-cant-multitask-stop-pretending-you-can?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[Fragmented attention produces fragmented work.&#xA;&#xA;When I split focus across tasks, I produce incomplete, low-quality output. Single-tasking changed that. I do deeper work, and I do more of it — no context-switching tax.&#xA;&#xA;Two habits made this stick.&#xA;&#xA;!--more--&#xA;&#xA;Cap your browser tabs at three. I used to keep dozens open — and used almost none of them. Three tabs forces a choice: what actually matters right now? I read one documentation page, close it, open the next. The constraint creates focus.&#xA;&#xA;Run every app in full screen. No dock. No red notification bubbles competing for your eye. I use two monitors — both apps full screen, side menus collapsed. Just the work, filling the frame.&#xA;&#xA;Attention is finite. Protect it like it is.&#xA;&#xA;!--emailsub--]]&gt;</description>
      <content:encoded><![CDATA[<p>Fragmented attention produces fragmented work.</p>

<p>When I split focus across tasks, I produce incomplete, low-quality output. Single-tasking changed that. I do deeper work, and I do more of it — no context-switching tax.</p>

<p>Two habits made this stick.</p>



<p><strong>Cap your browser tabs at three.</strong> I used to keep dozens open — and used almost none of them. Three tabs forces a choice: what actually matters right now? I read one documentation page, close it, open the next. The constraint creates focus.</p>

<p><strong>Run every app in full screen.</strong> No dock. No red notification bubbles competing for your eye. I use two monitors — both apps full screen, side menus collapsed. Just the work, filling the frame.</p>

<p>Attention is finite. Protect it like it is.</p>


]]></content:encoded>
      <guid>https://laxmena.com/most-people-cant-multitask-stop-pretending-you-can</guid>
      <pubDate>Thu, 02 Apr 2026 06:27:01 +0000</pubDate>
    </item>
    <item>
      <title>Hone vs. The 1 Billion Row Challenge</title>
      <link>https://laxmena.com/hone-vs-the-1-billion-row-challenge?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[1,000,000,000 rows of data. No hand-tuning. Just an agent, a benchmark, and a budget.&#xA;&#xA;The 1 Billion Row Challenge is simple on paper: read a file with 1B rows of weather station measurements, compute min/mean/max per station, as fast as possible. In Python, a naive solution takes minutes. The best human-optimized ones use memory-mapped files, multiprocessing, and numpy.&#xA;&#xA;I&#39;m not optimizing it by hand. I&#39;m giving it to Hone — and letting it figure it out.&#xA;&#xA;Hone is now on PyPI. Install it with pip install hone-ai.&#xA;&#xA;This is a living document. I&#39;ll update it as each run completes. Follow the code at laxmena/hone-1brc.&#xA;&#xA;!--more--&#xA;&#xA;---&#xA;&#xA;The Setup&#xA;&#xA;The challenge: Parse a 1B-row file. Each row: Hamburg;12.0. Compute min/mean/max per station. Print results sorted alphabetically.&#xA;&#xA;The metric: Wall-clock runtime in seconds. Lower is better.&#xA;&#xA;The constraints: Python standard library only. No numpy, no pandas, no third-party packages. Correctness must be preserved — output format and values must not change.&#xA;&#xA;The baseline:&#xA;&#xA;with open(filepath, &#34;r&#34;, encoding=&#34;utf-8&#34;) as f:&#xA;    for line in f:&#xA;        line = line.strip()&#xA;        sep = line.index(&#34;;&#34;)&#xA;        station = line[:sep]&#xA;        temp = float(line[sep + 1:])&#xA;        ...&#xA;&#xA;Simple. Correct. Slow. One thread, one line at a time, float() on every value.&#xA;&#xA;---&#xA;&#xA;Results at a Glance&#xA;&#xA;| Run | Model | Dataset | Baseline | Optimized | Improvement |&#xA;|-----|-------|---------|----------|-----------|-------------|&#xA;| 1 | Haiku | 1M rows | 0.546s | 0.471s | 13.7% |&#xA;| 2 | Haiku | 100M rows | 47.197s | 42.739s | 9.4% |&#xA;| 3 | Sonnet | 100M rows | 48.104s | 10.110s | 79% |&#xA;&#xA;---&#xA;&#xA;Episode 1: Haiku, 1M rows — 13.7% faster&#xA;&#xA;0.546s → 0.471s&#xA;&#xA;First run: claude-haiku-4-5, 1M rows, $5 budget, 50 max iterations.&#xA;&#xA;The 13.7% gain looks decent on paper. It isn&#39;t. The absolute numbers are tiny — we&#39;re talking 75 milliseconds. At this scale, Python startup time and OS disk caching dominate. The agent is optimizing noise, not the algorithm. Haiku made incremental tweaks but never found a structural breakthrough.&#xA;&#xA;Wrong dataset size. Move on.&#xA;&#xA;---&#xA;&#xA;Hone v1.2.0: --goal-file&#xA;&#xA;Episode 1 exposed a friction point. Pasting a long goal string into the terminal every run is error-prone and hard to version. For complex, multi-constraint goals it breaks down fast.&#xA;&#xA;I added --goal-file to Hone — pass a path to a plain text file, Hone reads the goal from there. Same idea as Karpathy&#39;s program.md in autoresearch. The goal now lives alongside the code, versioned in git.&#xA;&#xA;hone --goal-file program.md &#xA;     --bench &#34;python benchmark.py data/measurements100M.txt&#34; &#xA;     --files &#34;solution.py&#34; &#xA;     --optimize lower &#xA;     --score-pattern &#34;Time Taken:\s(\d+\.\d+)&#34; &#xA;     --budget 3.0 &#xA;     --max-iter 50 &#xA;     --model claude-haiku-4-5&#xA;&#xA;Live in v1.2.0. pip install --upgrade hone-ai.&#xA;&#xA;---&#xA;&#xA;Episode 2: Haiku, 100M rows — 9.4% faster&#xA;&#xA;47.197s → 42.739s&#xA;&#xA;10x harder dataset. Now I/O pressure actually matters — 4.5 seconds saved is a real signal.&#xA;&#xA;But Haiku still couldn&#39;t find the structural moves. It made safe, local edits — better buffering, minor parsing cleanup — and never stepped back to reconsider the architecture. No parallelism. No mmap. No integer parsing. It hit its ceiling.&#xA;&#xA;---&#xA;&#xA;Episode 3: Sonnet, 100M rows — 79% faster&#xA;&#xA;48.104s → 10.110s&#xA;&#xA;Same benchmark. Same constraints. One change: claude-haiku-4-5 → claude-sonnet-4-6.&#xA;&#xA;38 seconds saved. The agent didn&#39;t tune the baseline — it replaced it.&#xA;&#xA;What Sonnet actually did&#xA;&#xA;1. Text → Binary reads with mmap&#xA;&#xA;The baseline opens the file in text mode and reads line by line. Sonnet switched to binary mode with memory-mapped I/O — the OS maps the file directly into memory, eliminating repeated read syscalls.&#xA;&#xA;mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESSREAD)&#xA;chunk = mm[start:end]&#xA;&#xA;2. float() → integer arithmetic&#xA;&#xA;Every float() call in the baseline is expensive. Sonnet eliminated them entirely. Temperatures are stored as integers ×10 — 12.3 becomes 123. The decimal point is skipped by knowing its fixed position in the byte string. Division back to float happens only once, at output time.&#xA;&#xA;d0 = tb[-1] - 48           # last digit&#xA;val = (tb[0] - 48)  10 + d0   # b&#39;12.3&#39; → 123&#xA;&#xA;It also pre-built a lookup table for all valid temperature values (-99.9 to 99.9) to skip even manual parsing on the common case.&#xA;&#xA;3. Multiprocessing across all CPU cores&#xA;&#xA;The baseline is single-threaded. Sonnet split the file into cpucount() × 8 chunks, aligned each boundary to the next newline to avoid splitting rows, and ran each chunk in a separate process. Results merged at the end.&#xA;&#xA;numworkers = cpucount()&#xA;boundaries = findchunkboundaries(filepath, numworkers * 8)&#xA;with Pool(processes=numworkers) as pool:&#xA;    allstats = pool.map(processchunk, args)&#xA;&#xA;4. strip() + index() → partition()&#xA;&#xA;The baseline does line.strip() then line.index(&#34;;&#34;) — two passes. Sonnet used line.partition(b&#39;;&#39;) — one pass, station and temperature in a single call.&#xA;&#xA;Why Haiku couldn&#39;t find this&#xA;&#xA;Haiku made safe, local edits. It never stepped back to reconsider the architecture. Sonnet saw the whole picture: the bottleneck isn&#39;t any single line, it&#39;s the approach. Single-threaded text parsing doesn&#39;t scale. The winning move was to throw it out and start from a parallel, binary-aware design.&#xA;&#xA;Q: Does model choice matter more than iteration count?&#xA;&#xA;---&#xA;&#xA;What&#39;s Next&#xA;&#xA;100M rows, 79% faster. The real test is 1B rows — 10x again. Running next.&#xA;&#xA;---&#xA;&#xA;Updates appear here as experiments run. Subscribe below or follow via RSS._&#xA;&#xA;#engineering #hone #ai&#xA;&#xA;!--more--&#xA;]]&gt;</description>
      <content:encoded><![CDATA[<p>1,000,000,000 rows of data. No hand-tuning. Just an agent, a benchmark, and a budget.</p>

<p>The <a href="https://github.com/gunnarmorling/1brc">1 Billion Row Challenge</a> is simple on paper: read a file with 1B rows of weather station measurements, compute min/mean/max per station, as fast as possible. In Python, a naive solution takes minutes. The best human-optimized ones use memory-mapped files, multiprocessing, and numpy.</p>

<p>I&#39;m not optimizing it by hand. I&#39;m giving it to <a href="https://github.com/laxmena/hone">Hone</a> — and letting it figure it out.</p>

<p>Hone is now on PyPI. Install it with <code>pip install hone-ai</code>.</p>

<p>This is a living document. I&#39;ll update it as each run completes. Follow the code at <a href="https://github.com/laxmena/hone-1brc">laxmena/hone-1brc</a>.</p>



<hr/>

<h2 id="the-setup" id="the-setup">The Setup</h2>

<p><strong>The challenge:</strong> Parse a 1B-row file. Each row: <code>Hamburg;12.0</code>. Compute min/mean/max per station. Print results sorted alphabetically.</p>

<p><strong>The metric:</strong> Wall-clock runtime in seconds. Lower is better.</p>

<p><strong>The constraints:</strong> Python standard library only. No numpy, no pandas, no third-party packages. Correctness must be preserved — output format and values must not change.</p>

<p><strong>The baseline:</strong></p>

<pre><code class="language-python">with open(filepath, &#34;r&#34;, encoding=&#34;utf-8&#34;) as f:
    for line in f:
        line = line.strip()
        sep = line.index(&#34;;&#34;)
        station = line[:sep]
        temp = float(line[sep + 1:])
        ...
</code></pre>

<p>Simple. Correct. Slow. One thread, one line at a time, <code>float()</code> on every value.</p>

<hr/>

<h2 id="results-at-a-glance" id="results-at-a-glance">Results at a Glance</h2>

<table>
<thead>
<tr>
<th>Run</th>
<th>Model</th>
<th>Dataset</th>
<th>Baseline</th>
<th>Optimized</th>
<th>Improvement</th>
</tr>
</thead>

<tbody>
<tr>
<td>1</td>
<td>Haiku</td>
<td>1M rows</td>
<td>0.546s</td>
<td>0.471s</td>
<td>13.7%</td>
</tr>

<tr>
<td>2</td>
<td>Haiku</td>
<td>100M rows</td>
<td>47.197s</td>
<td>42.739s</td>
<td>9.4%</td>
</tr>

<tr>
<td>3</td>
<td>Sonnet</td>
<td>100M rows</td>
<td>48.104s</td>
<td>10.110s</td>
<td><strong>79%</strong></td>
</tr>
</tbody>
</table>

<hr/>

<h2 id="episode-1-haiku-1m-rows-13-7-faster" id="episode-1-haiku-1m-rows-13-7-faster">Episode 1: Haiku, 1M rows — 13.7% faster</h2>

<p><code>0.546s → 0.471s</code></p>

<p>First run: <code>claude-haiku-4-5</code>, 1M rows, $5 budget, 50 max iterations.</p>

<p>The 13.7% gain looks decent on paper. It isn&#39;t. The absolute numbers are tiny — we&#39;re talking 75 milliseconds. At this scale, Python startup time and OS disk caching dominate. The agent is optimizing noise, not the algorithm. Haiku made incremental tweaks but never found a structural breakthrough.</p>

<p>Wrong dataset size. Move on.</p>

<hr/>

<h2 id="hone-v1-2-0-goal-file" id="hone-v1-2-0-goal-file">Hone v1.2.0: <code>--goal-file</code></h2>

<p>Episode 1 exposed a friction point. Pasting a long goal string into the terminal every run is error-prone and hard to version. For complex, multi-constraint goals it breaks down fast.</p>

<p>I added <code>--goal-file</code> to Hone — pass a path to a plain text file, Hone reads the goal from there. Same idea as Karpathy&#39;s <code>program.md</code> in autoresearch. The goal now lives alongside the code, versioned in git.</p>

<pre><code class="language-bash">hone --goal-file program.md 
     --bench &#34;python benchmark.py data/measurements_100M.txt&#34; 
     --files &#34;solution.py&#34; 
     --optimize lower 
     --score-pattern &#34;Time Taken:\s*(\d+\.\d+)&#34; 
     --budget 3.0 
     --max-iter 50 
     --model claude-haiku-4-5
</code></pre>

<p>Live in <a href="https://github.com/laxmena/hone/commit/477a83d5050628355bf45ceded3807fea75b8ce6">v1.2.0</a>. <code>pip install --upgrade hone-ai</code>.</p>

<hr/>

<h2 id="episode-2-haiku-100m-rows-9-4-faster" id="episode-2-haiku-100m-rows-9-4-faster">Episode 2: Haiku, 100M rows — 9.4% faster</h2>

<p><code>47.197s → 42.739s</code></p>

<p>10x harder dataset. Now I/O pressure actually matters — 4.5 seconds saved is a real signal.</p>

<p>But Haiku still couldn&#39;t find the structural moves. It made safe, local edits — better buffering, minor parsing cleanup — and never stepped back to reconsider the architecture. No parallelism. No mmap. No integer parsing. It hit its ceiling.</p>

<hr/>

<h2 id="episode-3-sonnet-100m-rows-79-faster" id="episode-3-sonnet-100m-rows-79-faster">Episode 3: Sonnet, 100M rows — <strong>79% faster</strong></h2>

<p><code>48.104s → 10.110s</code></p>

<p>Same benchmark. Same constraints. One change: <code>claude-haiku-4-5</code> → <code>claude-sonnet-4-6</code>.</p>

<p>38 seconds saved. The agent didn&#39;t tune the baseline — it replaced it.</p>

<h3 id="what-sonnet-actually-did" id="what-sonnet-actually-did">What Sonnet actually did</h3>

<p><strong>1. Text → Binary reads with <code>mmap</code></strong></p>

<p>The baseline opens the file in text mode and reads line by line. Sonnet switched to binary mode with memory-mapped I/O — the OS maps the file directly into memory, eliminating repeated read syscalls.</p>

<pre><code class="language-python">mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
chunk = mm[start:end]
</code></pre>

<p><strong>2. <code>float()</code> → integer arithmetic</strong></p>

<p>Every <code>float()</code> call in the baseline is expensive. Sonnet eliminated them entirely. Temperatures are stored as integers ×10 — <code>12.3</code> becomes <code>123</code>. The decimal point is skipped by knowing its fixed position in the byte string. Division back to float happens only once, at output time.</p>

<pre><code class="language-python">d0 = tb[-1] - 48           # last digit
val = (tb[0] - 48) * 10 + d0   # b&#39;12.3&#39; → 123
</code></pre>

<p>It also pre-built a lookup table for all valid temperature values (<code>-99.9</code> to <code>99.9</code>) to skip even manual parsing on the common case.</p>

<p><strong>3. Multiprocessing across all CPU cores</strong></p>

<p>The baseline is single-threaded. Sonnet split the file into <code>cpu_count() × 8</code> chunks, aligned each boundary to the next newline to avoid splitting rows, and ran each chunk in a separate process. Results merged at the end.</p>

<pre><code class="language-python">num_workers = cpu_count()
boundaries = find_chunk_boundaries(filepath, num_workers * 8)
with Pool(processes=num_workers) as pool:
    all_stats = pool.map(process_chunk, args)
</code></pre>

<p><strong>4. <code>strip()</code> + <code>index()</code> → <code>partition()</code></strong></p>

<p>The baseline does <code>line.strip()</code> then <code>line.index(&#34;;&#34;)</code> — two passes. Sonnet used <code>line.partition(b&#39;;&#39;)</code> — one pass, station and temperature in a single call.</p>

<h3 id="why-haiku-couldn-t-find-this" id="why-haiku-couldn-t-find-this">Why Haiku couldn&#39;t find this</h3>

<p>Haiku made safe, local edits. It never stepped back to reconsider the architecture. Sonnet saw the whole picture: the bottleneck isn&#39;t any single line, it&#39;s the approach. Single-threaded text parsing doesn&#39;t scale. The winning move was to throw it out and start from a parallel, binary-aware design.</p>

<p><strong>Q: Does model choice matter more than iteration count?</strong></p>

<hr/>

<h2 id="what-s-next" id="what-s-next">What&#39;s Next</h2>

<p>100M rows, 79% faster. The real test is 1B rows — 10x again. Running next.</p>

<hr/>

<p><em>Updates appear here as experiments run. Subscribe below or follow via <a href="https://write.as/laxmena/feed/">RSS</a>.</em></p>

<p><a href="https://laxmena.com/tag:engineering" class="hashtag"><span>#</span><span class="p-category">engineering</span></a> <a href="https://laxmena.com/tag:hone" class="hashtag"><span>#</span><span class="p-category">hone</span></a> <a href="https://laxmena.com/tag:ai" class="hashtag"><span>#</span><span class="p-category">ai</span></a></p>


]]></content:encoded>
      <guid>https://laxmena.com/hone-vs-the-1-billion-row-challenge</guid>
      <pubDate>Wed, 25 Mar 2026 04:06:42 +0000</pubDate>
    </item>
    <item>
      <title>I Built a Tool That Optimizes Code While You Sleep</title>
      <link>https://laxmena.com/i-built-a-tool-that-optimizes-code-while-you-sleep?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[A few weeks ago, I watched a Karpathy talk where he described running an agentic loop to auto-tune LLM fine-tuning pipelines. The core idea was simple: give the agent a goal, a way to measure progress, and let it iterate autonomously until it gets there.&#xA;&#xA;I couldn&#39;t stop thinking about it.&#xA;&#xA;Not because of the fine-tuning use case — but because the pattern felt universally useful. Most software has something you want to improve and a way to measure it. Why are we still doing the iteration loop by hand?&#xA;&#xA;So I built Hone.&#xA;&#xA;!--more--&#xA;&#xA;What Hone Does&#xA;&#xA;Hone is a CLI tool. You give it three things:&#xA;&#xA;A goal, in plain English&#xA;&#xA;A file or directory to optimize&#xA;&#xA;A benchmark command that outputs a number&#xA;&#xA;Then you leave.&#xA;&#xA;Hone runs a loop: it asks an LLM what to try next, applies the changes, runs your benchmark, and decides whether to keep the result or revert it. It logs every iteration — the score, the diff, and the agent&#39;s reasoning — and stops when it hits your target or you tell it to.&#xA;&#xA;hone &#34;Optimize processlogs.py to run under 0.02 seconds&#34; &#xA;     --bench &#34;python benchlogs.py&#34; &#xA;     --files &#34;processlogs.py&#34; &#xA;     --optimize lower &#xA;     --target 0.02 &#xA;     --budget 2.0&#xA;&#xA;That&#39;s the entire interface.&#xA;&#xA;---&#xA;&#xA;Experiment 1: The Log Parser&#xA;&#xA;The first real test was a deliberately naive Python log parser. The task: analyze 150,000 lines of server logs and return the top 3 most-visited endpoints with unique IP counts.&#xA;&#xA;The baseline code was the kind you&#39;d write in an interview warm-up: readlines() into memory, a list for uniqueness checking (O(n) per insert), a regex match on every line. It took 1.54 seconds.&#xA;&#xA;I set a target of 0.02 seconds — roughly 75x faster — and launched Hone with a $2 budget.&#xA;&#xA;Here&#39;s what happened over 20 iterations:&#xA;&#xA;| Iter | Score | What the agent did |&#xA;|------|-------|--------------------|&#xA;| 1–4 | 0.8s → 0.4s | Replaced list with set for O(1) uniqueness, pre-bound set.add to skip attribute lookup overhead |&#xA;| 5–9 | 0.4s → 0.15s | Switched from readlines() to streaming with f, dropped unnecessary string allocations |&#xA;| 10–14 | 0.15s → 0.09s | Compiled regex outside the loop, switched from re.match to re.search with anchored pattern |&#xA;| 15–17 | 0.09s → 0.07s | Plateaued. Agent recognized it had hit the ceiling of single-threaded Python looping. |&#xA;| 18–20 | 0.07s → 0.037s | Changed the rules entirely. Abandoned line-by-line parsing. Read the file as a raw binary blob. Deployed re.findall() over the entire content in one pass. |&#xA;&#xA;The final move was the interesting one. The agent didn&#39;t just tune the existing approach — it recognized the approach itself was the bottleneck and replaced it. That pivot happened at iteration 18, after the agent wrote in its reasoning:&#xA;&#xA;  &#34;The real bottleneck is the Python loop and split() calls. Try using a compiled regex to extract the endpoint in one operation across the entire file.&#34;&#xA;&#xA;Final result: 1.54s → 0.037s. A 41x speedup. Autonomously.&#xA;&#xA;It didn&#39;t hit the 0.02 target — that&#39;s likely beyond what single-threaded Python can do on this task without going to C extensions. But a 41x improvement for $1.84 in API costs is a real result.&#xA;&#xA;---&#xA;&#xA;Experiment 2: Nearest Driver Dispatch&#xA;&#xA;The second experiment was closer to production code. The problem: given a set of riders and a pool of drivers, find the nearest driver for each rider using haversine distance.&#xA;&#xA;The baseline was an O(R × D) brute-force loop — calculate the full haversine distance between every rider and every driver. With 500 riders and 1,000 drivers, that&#39;s 500,000 distance calculations per call. Baseline: 2.18 seconds.&#xA;&#xA;Run 1 — I launched Hone with no hints. Just: &#34;optimize this to run faster.&#34;&#xA;&#xA;The agent went straight for spatial indexing. It built a grid over the geographic area, bucketed drivers into cells, and used Manhattan distance pre-filtering to eliminate distant candidates before running haversine. It also replaced the standard math module haversine with a vectorized approximation valid for short distances.&#xA;&#xA;Result: 0.1496 seconds. A 14.6x speedup.&#xA;&#xA;Run 2 — I ran Hone again on the output from Run 1.&#xA;&#xA;This is where it got interesting. The agent looked at the already-optimized code and found something the previous run missed: the grid search still checked every driver in candidate cells, even after it had already found a close one.&#xA;&#xA;The fix: stop searching the moment you find a driver within an acceptable radius. Expand the search radius incrementally — start small, grow outward — instead of checking all candidates at once.&#xA;&#xA;  &#34;The algorithm beats the data structure. Grid resolution barely matters. Early termination dominates.&#34;&#xA;&#xA;Result: 0.069 seconds. Another 2.1x on top of an already fast baseline.&#xA;&#xA;Two runs, $3 total, brute-force O(R×D) → smart early-termination spatial search. The agent arrived at an approach that a senior engineer would recognize as correct — not by knowing the algorithm upfront, but by observing what the benchmark rewarded.&#xA;&#xA;---&#xA;&#xA;What I Learned&#xA;&#xA;The benchmark is everything. Hone is only as good as your measurement. If your benchmark is slow to run, the loop is slow. If it doesn&#39;t capture what you actually care about, the agent will optimize the wrong thing. The one thing you must get right before you start is: &#34;does this number actually reflect what I want?&#34;&#xA;&#xA;The agent is a good low-level optimizer. It reliably finds the obvious wins: wrong data structures, redundant computations, missed language primitives. These are also the wins that take a human the most time — not because they&#39;re hard to understand, but because you have to actually sit down and do them.&#xA;&#xA;It surprises you at the edges. The log parser pivot from line-by-line to whole-file regex wasn&#39;t something I would have thought to suggest in the initial prompt. It emerged from the agent hitting a wall and reasoning about why_ it had hit a wall. That&#39;s the behavior that makes agentic loops interesting.&#xA;&#xA;The conversation thread is the memory. The most important architectural decision in Hone was keeping the LLM conversation alive across iterations. The agent doesn&#39;t just see the current score — it sees everything it tried, what worked, and what was reverted. That&#39;s what allows the pivot at iteration 18. Without it, the agent would start fresh each time and repeat the same early optimizations.&#xA;&#xA;Cost is low. Time savings are high. Both experiments ran under $4. The engineering time to achieve the same results manually — writing hypotheses, applying changes, running benchmarks, reverting dead ends — would have been hours. The ROI on agentic loops is already real, and we&#39;re at the beginning.&#xA;&#xA;---&#xA;&#xA;What&#39;s Next&#xA;&#xA;Hone v0 is rough. There&#39;s no sandbox for shell commands, no git-based snapshots, no dry-run mode. These are on the list.&#xA;&#xA;More interesting to me is expanding the use cases. The same loop that optimizes a log parser can optimize:&#xA;&#xA;LLM prompts against an eval suite (highest impact use case)&#xA;RAG pipelines against a retrieval benchmark&#xA;API costs against a quality-constrained spend target&#xA;&#xA;The pattern is the same. The benchmark changes. Hone doesn&#39;t care.&#xA;&#xA;If you want to try it:&#xA;&#xA;git clone https://github.com/laxmena/hone&#xA;cd hone &amp;&amp; pip install -e .&#xA;&#xA;And if you have a benchmark that Hone should try — I want to hear about it.&#xA;&#xA;#engineering #ai&#xA;&#xA;---&#xA;&#xA;!--more--&#xA;]]&gt;</description>
      <content:encoded><![CDATA[<p>A few weeks ago, I watched a <a href="https://x.com/karpathy">Karpathy talk</a> where he described running an agentic loop to auto-tune LLM fine-tuning pipelines. The core idea was simple: give the agent a goal, a way to measure progress, and let it iterate autonomously until it gets there.</p>

<p>I couldn&#39;t stop thinking about it.</p>

<p>Not because of the fine-tuning use case — but because the <em>pattern</em> felt universally useful. Most software has something you want to improve and a way to measure it. Why are we still doing the iteration loop by hand?</p>

<p>So I built <a href="https://github.com/laxmena/hone">Hone</a>.</p>



<h2 id="what-hone-does" id="what-hone-does">What Hone Does</h2>

<p>Hone is a CLI tool. You give it three things:</p>
<ol><li><p>A goal, in plain English</p></li>

<li><p>A file or directory to optimize</p></li>

<li><p>A benchmark command that outputs a number</p></li></ol>

<p>Then you leave.</p>

<p>Hone runs a loop: it asks an LLM what to try next, applies the changes, runs your benchmark, and decides whether to keep the result or revert it. It logs every iteration — the score, the diff, and the agent&#39;s reasoning — and stops when it hits your target or you tell it to.</p>

<pre><code class="language-bash">hone &#34;Optimize process_logs.py to run under 0.02 seconds&#34; 
     --bench &#34;python bench_logs.py&#34; 
     --files &#34;process_logs.py&#34; 
     --optimize lower 
     --target 0.02 
     --budget 2.0
</code></pre>

<p>That&#39;s the entire interface.</p>

<hr/>

<h2 id="experiment-1-the-log-parser" id="experiment-1-the-log-parser">Experiment 1: The Log Parser</h2>

<p>The first real test was a deliberately naive Python log parser. The task: analyze 150,000 lines of server logs and return the top 3 most-visited endpoints with unique IP counts.</p>

<p>The baseline code was the kind you&#39;d write in an interview warm-up: <code>readlines()</code> into memory, a list for uniqueness checking (O(n) per insert), a regex match on every line. It took <strong>1.54 seconds</strong>.</p>

<p>I set a target of 0.02 seconds — roughly 75x faster — and launched Hone with a $2 budget.</p>

<p>Here&#39;s what happened over 20 iterations:</p>

<table>
<thead>
<tr>
<th>Iter</th>
<th>Score</th>
<th>What the agent did</th>
</tr>
</thead>

<tbody>
<tr>
<td>1–4</td>
<td>0.8s → 0.4s</td>
<td>Replaced list with <code>set</code> for O(1) uniqueness, pre-bound <code>set.add</code> to skip attribute lookup overhead</td>
</tr>

<tr>
<td>5–9</td>
<td>0.4s → 0.15s</td>
<td>Switched from <code>readlines()</code> to streaming with <code>f</code>, dropped unnecessary string allocations</td>
</tr>

<tr>
<td>10–14</td>
<td>0.15s → 0.09s</td>
<td>Compiled regex outside the loop, switched from <code>re.match</code> to <code>re.search</code> with anchored pattern</td>
</tr>

<tr>
<td>15–17</td>
<td>0.09s → 0.07s</td>
<td>Plateaued. Agent recognized it had hit the ceiling of single-threaded Python looping.</td>
</tr>

<tr>
<td>18–20</td>
<td>0.07s → <strong>0.037s</strong></td>
<td><strong>Changed the rules entirely.</strong> Abandoned line-by-line parsing. Read the file as a raw binary blob. Deployed <code>re.findall()</code> over the entire content in one pass.</td>
</tr>
</tbody>
</table>

<p>The final move was the interesting one. The agent didn&#39;t just tune the existing approach — it recognized the approach itself was the bottleneck and replaced it. That pivot happened at iteration 18, after the agent wrote in its reasoning:</p>

<blockquote><p><em>“The real bottleneck is the Python loop and split() calls. Try using a compiled regex to extract the endpoint in one operation across the entire file.”</em></p></blockquote>

<p><strong>Final result: 1.54s → 0.037s. A 41x speedup. Autonomously.</strong></p>

<p>It didn&#39;t hit the 0.02 target — that&#39;s likely beyond what single-threaded Python can do on this task without going to C extensions. But a 41x improvement for $1.84 in API costs is a real result.</p>

<hr/>

<h2 id="experiment-2-nearest-driver-dispatch" id="experiment-2-nearest-driver-dispatch">Experiment 2: Nearest Driver Dispatch</h2>

<p>The second experiment was closer to production code. The problem: given a set of riders and a pool of drivers, find the nearest driver for each rider using haversine distance.</p>

<p>The baseline was an O(R × D) brute-force loop — calculate the full haversine distance between every rider and every driver. With 500 riders and 1,000 drivers, that&#39;s 500,000 distance calculations per call. Baseline: <strong>2.18 seconds</strong>.</p>

<p><strong>Run 1</strong> — I launched Hone with no hints. Just: <em>“optimize this to run faster.”</em></p>

<p>The agent went straight for spatial indexing. It built a grid over the geographic area, bucketed drivers into cells, and used Manhattan distance pre-filtering to eliminate distant candidates before running haversine. It also replaced the standard <code>math</code> module haversine with a vectorized approximation valid for short distances.</p>

<p>Result: <strong>0.1496 seconds. A 14.6x speedup.</strong></p>

<p><strong>Run 2</strong> — I ran Hone again on the output from Run 1.</p>

<p>This is where it got interesting. The agent looked at the already-optimized code and found something the previous run missed: the grid search still checked every driver in candidate cells, even after it had already found a close one.</p>

<p>The fix: stop searching the moment you find a driver within an acceptable radius. Expand the search radius incrementally — start small, grow outward — instead of checking all candidates at once.</p>

<blockquote><p><em>“The algorithm beats the data structure. Grid resolution barely matters. Early termination dominates.”</em></p></blockquote>

<p>Result: <strong>0.069 seconds. Another 2.1x on top of an already fast baseline.</strong></p>

<p>Two runs, $3 total, brute-force O(R×D) → smart early-termination spatial search. The agent arrived at an approach that a senior engineer would recognize as correct — not by knowing the algorithm upfront, but by observing what the benchmark rewarded.</p>

<hr/>

<h2 id="what-i-learned" id="what-i-learned">What I Learned</h2>

<p><strong>The benchmark is everything.</strong> Hone is only as good as your measurement. If your benchmark is slow to run, the loop is slow. If it doesn&#39;t capture what you actually care about, the agent will optimize the wrong thing. The one thing you must get right before you start is: “does this number actually reflect what I want?”</p>

<p><strong>The agent is a good low-level optimizer.</strong> It reliably finds the obvious wins: wrong data structures, redundant computations, missed language primitives. These are also the wins that take a human the most time — not because they&#39;re hard to understand, but because you have to actually sit down and do them.</p>

<p><strong>It surprises you at the edges.</strong> The log parser pivot from line-by-line to whole-file regex wasn&#39;t something I would have thought to suggest in the initial prompt. It emerged from the agent hitting a wall and reasoning about <em>why</em> it had hit a wall. That&#39;s the behavior that makes agentic loops interesting.</p>

<p><strong>The conversation thread is the memory.</strong> The most important architectural decision in Hone was keeping the LLM conversation alive across iterations. The agent doesn&#39;t just see the current score — it sees everything it tried, what worked, and what was reverted. That&#39;s what allows the pivot at iteration 18. Without it, the agent would start fresh each time and repeat the same early optimizations.</p>

<p><strong>Cost is low. Time savings are high.</strong> Both experiments ran under $4. The engineering time to achieve the same results manually — writing hypotheses, applying changes, running benchmarks, reverting dead ends — would have been hours. The ROI on agentic loops is already real, and we&#39;re at the beginning.</p>

<hr/>

<h2 id="what-s-next" id="what-s-next">What&#39;s Next</h2>

<p>Hone v0 is rough. There&#39;s no sandbox for shell commands, no git-based snapshots, no dry-run mode. These are on the list.</p>

<p>More interesting to me is expanding the use cases. The same loop that optimizes a log parser can optimize:</p>
<ul><li><strong>LLM prompts</strong> against an eval suite (highest impact use case)</li>
<li><strong>RAG pipelines</strong> against a retrieval benchmark</li>
<li><strong>API costs</strong> against a quality-constrained spend target</li></ul>

<p>The pattern is the same. The benchmark changes. Hone doesn&#39;t care.</p>

<p>If you want to try it:</p>

<pre><code class="language-bash">git clone https://github.com/laxmena/hone
cd hone &amp;&amp; pip install -e .
</code></pre>

<p>And if you have a benchmark that Hone should try — I want to hear about it.</p>

<p><a href="https://laxmena.com/tag:engineering" class="hashtag"><span>#</span><span class="p-category">engineering</span></a> <a href="https://laxmena.com/tag:ai" class="hashtag"><span>#</span><span class="p-category">ai</span></a></p>

<hr/>


]]></content:encoded>
      <guid>https://laxmena.com/i-built-a-tool-that-optimizes-code-while-you-sleep</guid>
      <pubDate>Tue, 24 Mar 2026 03:06:12 +0000</pubDate>
    </item>
    <item>
      <title>The Day You Became A Better Writer</title>
      <link>https://laxmena.com/the-day-you-became-a-better-writer?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[In 2007, Scott Adams — creator of Dilbert — published a short blog post on writing. Naval Ravikant thought it was worth adding to his recommended reading list in the Almanack of Naval Ravikant.&#xA;&#xA;There&#39;s one problem. Typepad, the blogging platform that hosted it, shut down permanently on September 30, 2025. The post disappeared with it.&#xA;&#xA;I tracked it down through the Internet Archive. You can read the original here.&#xA;&#xA;This post is my attempt to make it accessible — and to add something new.&#xA;&#xA;!--more--&#xA;&#xA;---&#xA;&#xA;What Adams said&#xA;&#xA;Adams opens with a claim: he went from bad writer to good writer after a single one-day course in business writing. Then he gives you the whole course in under 200 words.&#xA;&#xA;The core idea is simple. Simple writing is persuasive. A tight five-sentence argument beats a sprawling hundred-sentence one. Every time.&#xA;&#xA;Here are his rules, distilled:&#xA;&#xA;The Day You Became A Better Writer — infographic&#xA;&#xA;---&#xA;&#xA;My additions&#xA;&#xA;Adams covers the sentence level well. These extend his thinking to structure.&#xA;&#xA;7. Front-load your point. State the conclusion first, then support it. Don&#39;t make the reader work through the argument before knowing why it matters.&#xA;&#xA;8. One idea per paragraph. Adams says one thought per sentence. The same logic applies one level up. If a paragraph is doing two jobs, split it.&#xA;&#xA;---&#xA;&#xA;Steal this prompt&#xA;&#xA;If you use LLMs to help draft or edit writing, here&#39;s a prompt you can drop into your workflow. It distills everything above into instructions the model will actually follow.&#xA;&#xA;You are a writing assistant that helps produce clear, persuasive, and readable text.&#xA;&#xA;Follow these principles when writing or editing:&#xA;&#xA;Keep it simple. A short, clear argument is more persuasive than a long, complex one.&#xA;Cut extra words. If a word doesn&#39;t add meaning, remove it.&#xA;Choose potent words. Prefer the specific and vivid over the generic.&#xA;Make the first sentence earn attention. It should create curiosity or make a bold claim.&#xA;Write short sentences. One thought per sentence.&#xA;Use active voice. Put the actor before the action.&#xA;Front-load the point. State the conclusion first, then support it.&#xA;One idea per paragraph. If a paragraph is doing two jobs, split it.&#xA;&#xA;When editing, flag sentences that violate these rules and suggest alternatives.&#xA;&#xA;---&#xA;&#xA;Good writing is good thinking made visible. Adams knew this in 2007. It hasn&#39;t changed.&#xA;&#xA;---&#xA;&#xA;All original ideas referenced here belong to Scott Adams. This post exists to preserve and extend his thinking, not to replace it. Read the original._&#xA;&#xA;writing&#xA;&#xA;!--more--&#xA;]]&gt;</description>
      <content:encoded><![CDATA[<p>In 2007, Scott Adams — creator of Dilbert — published a short blog post on writing. Naval Ravikant thought it was worth adding to his recommended reading list in the Almanack of Naval Ravikant.</p>

<p>There&#39;s one problem. Typepad, the blogging platform that hosted it, shut down permanently on September 30, 2025. The post disappeared with it.</p>

<p>I tracked it down through the Internet Archive. You can read the <a href="https://web.archive.org/web/20240302003157/https://dilbertblog.typepad.com/the_dilbert_blog/2007/06/the_day_you_bec.html">original here</a>.</p>

<p>This post is my attempt to make it accessible — and to add something new.</p>



<hr/>

<h2 id="what-adams-said" id="what-adams-said">What Adams said</h2>

<p>Adams opens with a claim: he went from bad writer to good writer after a single one-day course in business writing. Then he gives you the whole course in under 200 words.</p>

<p>The core idea is simple. Simple writing is persuasive. A tight five-sentence argument beats a sprawling hundred-sentence one. Every time.</p>

<p>Here are his rules, distilled:</p>

<p><img src="https://raw.githubusercontent.com/laxmena/blog-assets/refs/heads/main/images/writing_infographic.png" alt="The Day You Became A Better Writer — infographic"/></p>

<hr/>

<h2 id="my-additions" id="my-additions">My additions</h2>

<p>Adams covers the sentence level well. These extend his thinking to structure.</p>

<p><strong>7. Front-load your point.</strong> State the conclusion first, then support it. Don&#39;t make the reader work through the argument before knowing why it matters.</p>

<p><strong>8. One idea per paragraph.</strong> Adams says one thought per sentence. The same logic applies one level up. If a paragraph is doing two jobs, split it.</p>

<hr/>

<h2 id="steal-this-prompt" id="steal-this-prompt">Steal this prompt</h2>

<p>If you use LLMs to help draft or edit writing, here&#39;s a prompt you can drop into your workflow. It distills everything above into instructions the model will actually follow.</p>

<pre><code>You are a writing assistant that helps produce clear, persuasive, and readable text.

Follow these principles when writing or editing:

- Keep it simple. A short, clear argument is more persuasive than a long, complex one.
- Cut extra words. If a word doesn&#39;t add meaning, remove it.
- Choose potent words. Prefer the specific and vivid over the generic.
- Make the first sentence earn attention. It should create curiosity or make a bold claim.
- Write short sentences. One thought per sentence.
- Use active voice. Put the actor before the action.
- Front-load the point. State the conclusion first, then support it.
- One idea per paragraph. If a paragraph is doing two jobs, split it.

When editing, flag sentences that violate these rules and suggest alternatives.
</code></pre>

<hr/>

<p>Good writing is good thinking made visible. Adams knew this in 2007. It hasn&#39;t changed.</p>

<hr/>

<p><em>All original ideas referenced here belong to Scott Adams. This post exists to preserve and extend his thinking, not to replace it. Read the [original](<a href="https://web.archive.org/web/20240302003157/https://dilbertblog.typepad.com/the">https://web.archive.org/web/20240302003157/https://dilbertblog.typepad.com/the</a></em>dilbert<em>blog/2007/06/the</em>day<em>you</em>bec.html)._</p>

<p><a href="https://laxmena.com/tag:writing" class="hashtag"><span>#</span><span class="p-category">writing</span></a></p>


]]></content:encoded>
      <guid>https://laxmena.com/the-day-you-became-a-better-writer</guid>
      <pubDate>Sun, 22 Mar 2026 02:25:45 +0000</pubDate>
    </item>
    <item>
      <title>RiskChain: The Messy Middle: Building a Risk Graph from Scratch</title>
      <link>https://laxmena.com/riskchain-the-messy-middle-building-a-risk-graph-from-scratch?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[An ongoing weekend project documenting the journey of uncovering hidden connections in corporate financial filings—the stumbles, the learnings, the &#39;aha!&#39; moments, and everything in between. Started January 2025.&#xA;&#xA;---&#xA;&#xA;What is RiskChain?&#xA;&#xA;The core idea is simple but ambitious: find hidden connections and risk trails that aren&#39;t immediately obvious when you&#39;re just reading through a 10-K filing.&#xA;&#xA;!--more--&#xA;&#xA;Instead of treating each financial document as an isolated artifact, I&#39;m building a system to:&#xA;&#xA;Extract risk factors from 10-K filings (2004-2025) across 75 companies&#xA;Embed and connect these risks to find non-obvious relationships&#xA;Build a graph that reveals risk clusters, patterns, and &#34;trails&#34; that could signal systemic weaknesses or early warning signs&#xA;&#xA;Why 10-K filings? Because companies are required to disclose risks in specific sections (Item 1 and Item 1a), and there&#39;s a decade+ of structured data just sitting there.&#xA;&#xA;---&#xA;&#xA;The Vision&#xA;&#xA;Here&#39;s the full pipeline I&#39;m building toward:&#xA;&#xA;[Raw Financial Data]&#xA;  ├── SEC Filings (10-K/Q) ── News Articles ── Earnings Transcripts ── Other Reports&#xA;          │&#xA;          ▼&#xA;[1. Ingestion &amp; Chunking]&#xA;  → Parse documents (PDF/HTML) → Split into sentences → Group into ~500-word chunks&#xA;          │&#xA;          ▼&#xA;[2. Risk Extraction]&#xA;  → Use Gemini Flash per chunk → Extract 3-5 specific risk factors + severity&#xA;          │&#xA;          ▼&#xA;[3. Storage &amp; Embeddings]&#xA;  → SQLite DB (with sqlite-vec) → Embed risk labels (embedding-gemma-300m) → Deduplicate similar risks&#xA;          │&#xA;          ▼&#xA;[4. Graph Construction]&#xA;  → Nodes = unique risks&#xA;  → Edges = &#xA;      ├─ Semantic similarity (embeddings)&#xA;      └─ Statistical co-occurrence (PMI)&#xA;          │&#xA;          ▼&#xA;[5. Hierarchical Clustering]&#xA;  → Apply Leiden algorithm (Surprise function) → Build risk hierarchy tree&#xA;  → Compute novelty scores for under-explored areas&#xA;          │&#xA;          ▼&#xA;[6. CLI / Interface Layer]&#xA;  → Persistent server for fast queries&#xA;  → Commands: searchrisks, browsetree, crossreportrisks, etc.&#xA;          │&#xA;          ▼&#xA;[7. Agent Workflow (Claude / similar)]&#xA;  ├── Stage 1: Ideation ── Browse tree → Propose novel risk chains (novelty bias)&#xA;  ├── Stage 2: Research ── Dive into chunks → Extract &amp; order excerpts&#xA;  └── Stage 3: Output ── Generate RiskChain (visual trail with edges + narrative)&#xA;          │&#xA;          ▼&#xA;[8. Presentation &amp; Action]&#xA;  → Web dashboard / exported report&#xA;  → Visual graph + highlighted excerpts + suggested hedges / alerts&#xA;  → Human review → Iterate via feedback&#xA;&#xA;It&#39;s ambitious. It&#39;s probably overambitious. But that&#39;s the goal.&#xA;&#xA;---&#xA;&#xA;Current Status&#xA;&#xA;Phase: 2 - Chunking Strategy ✓&#xA;Progress: Data downloaded → Chunking complete → Ready for Risk Extraction&#xA;&#xA;---&#xA;&#xA;Stay Updated&#xA;&#xA;I&#39;m documenting this journey every weekend—the wins, the blockers, the learnings. If you want regular updates on how RiskChain develops, subscribe below to get new posts delivered to your inbox.&#xA;&#xA;!--more--&#xA;&#xA;---&#xA;&#xA;Progress Log&#xA;&#xA;Weekend 1 | Jan 18, 2025 | Phase 1: Download Script ✓&#xA;&#xA;What I built:&#xA;Downloaded 10-K filings for 75 companies from 2004-2025 using the Python edgartools library. Curated a list of significant companies (including ones that went bankrupt in 2008—why not?). Got the script working and only extracting the relevant sections (Item 1, Item 7, Item 8) to keep things lean.&#xA;&#xA;The messy parts (aka real life):&#xA;I initially tried sec-edgar-downloader to connect to SEC and download. Spent way too much time on this approach, got stuck in the data cleaning rabbit hole, and realized I was losing sight of the actual goal. The real issue? Many of the 10-K filings before the SEC standardized their item categorization didn&#39;t play nice with the tool.&#xA;&#xA;  Lesson learned: when you&#39;re iterating, it&#39;s okay to abandon the &#34;perfect&#34; approach for one that ships faster.&#xA;&#xA;Then I switched to edgartools (also known as edgar). This library gave me more flexibility, though the documentation still wasn&#39;t intuitive for my specific use case. But instead of giving up, I dug into the source code. That&#39;s when things clicked. Sometimes the best learning comes from reading other people&#39;s code instead of waiting for docs to explain everything.&#xA;&#xA;The &#39;aha!&#39; moment:&#xA;&#xA;  My wife helped me understand what Item 1, Item 1a, Item 7, and Item 8 actually mean in a 10-K filing. She translated the financial jargon into plain English, and suddenly the document structure made sense. Having someone who can bridge the domain knowledge gap is invaluable. I realized I was building this in a foreign domain—finance is not my native language, and that&#39;s okay.&#xA;&#xA;What blocked me:&#xA;&#xA;Figuring out the right tool for downloading (sec-edgar-downloader vs edgartools vs rolling my own)&#xA;Understanding that parsing 10-K files is genuinely harder than it looks (inconsistent structures across years, weird formatting, embedded tables)&#xA;&#xA;Next up: Phase 2: Chunking strategy. Need to figure out how to split these documents intelligently for downstream LLM tasks.&#xA;&#xA;---&#xA;&#xA;Weekend 2 | Jan 23, 2025 | Phase 2: Chunking Strategy ✓&#xA;&#xA;What I built:&#xA;Implemented chunking using wtpsplitter and stored all chunks as markdown files with YAML frontmatter metadata (ticker, filing date, company name, chunk ID, item section). Now sitting on several thousand chunks, each \~1000 characters max, ready for extraction.&#xA;&#xA;The messy parts (aka real life):&#xA;I tried two chunking strategies: RecursiveChunker and wtpsplitter. RecursiveChunker felt like brute force—just splitting on token counts. But wtpsplitter was smarter; it respects sentence boundaries and creates more semantically coherent chunks.&#xA;&#xA;Storing these as markdown files locally feels like a step backward (shouldn&#39;t I be using a database?), but honestly, it&#39;s perfect for iteration. I can inspect the chunks, debug the metadata, and understand what&#39;s happening before I add the complexity of a full DB setup.&#xA;&#xA;The &#39;aha!&#39; moment:&#xA;&#xA;  Chunk quality matters way more than I initially thought. The way you split text directly impacts whether an LLM can extract meaningful risk factors later. Sentence-aware chunking beats token-counting brutality. This made me reconsider the whole &#34;let me jump straight to a database&#34; instinct. Sometimes you need to slow down and get the fundamentals right first.&#xA;&#xA;What blocked me:&#xA;&#xA;Deciding between chunking strategies (trial and error on a few approaches)&#xA;Understanding the tradeoff between local file storage and &#34;proper&#34; database setup (spoiler: local storage is fine for now)&#xA;Realizing I was overthinking this phase when the real value comes next&#xA;&#xA;Next up: Phase 3: Risk Extraction. I&#39;ll iterate through each chunk and use Claude/Gemini to extract 3-5 risk factors per chunk. This is where the actual signal starts emerging.&#xA;&#xA;---&#xA;&#xA;Why This Matters (and Why I&#39;m Excited)&#xA;&#xA;Most financial analysis tools treat risks as isolated items. &#34;Company X faces supply chain risk.&#34; &#34;Company Y has regulatory exposure.&#34; But what if you could see that 40 companies in the industrial sector all mention the same emerging regulatory risk, and 3 of them went bankrupt 2 years later?&#xA;&#xA;That&#39;s the thesis here. Hidden connections. Patterns that emerge when you look at scale.&#xA;&#xA;Also, I&#39;m learning a ton: SEC filing structures, chunking strategies, embedding models, graph theory, the Leiden algorithm... This is weekend learning on steroids.&#xA;&#xA;---&#xA;&#xA;Updates added weekly (weekends permitting). Check back for new learnings, blockers, and wins.&#xA;&#xA;---&#xA;&#xA;Resources &amp; References&#xA;&#xA;Inspiration: Syntopic Reading with Claude — The original spark for connecting documents at scale&#xA;Graph Clustering: Leiden Algorithm Documentation — For hierarchical risk clustering&#xA;SEC Data Tool: edgartools (edgar) — Python library for downloading SEC filings&#xA;Alternative Tool: sec-edgar-downloader — The tool I explored first (works well for recent filings; struggled with older 10-Ks before SEC standardization)&#xA;&#xA;#engineering #ai]]&gt;</description>
      <content:encoded><![CDATA[<p><em>An ongoing weekend project documenting the journey of uncovering hidden connections in corporate financial filings—the stumbles, the learnings, the &#39;aha!&#39; moments, and everything in between. Started January 2025.</em></p>

<hr/>

<h3 id="what-is-riskchain" id="what-is-riskchain">What is RiskChain?</h3>

<p>The core idea is simple but ambitious: <strong>find hidden connections and risk trails that aren&#39;t immediately obvious</strong> when you&#39;re just reading through a 10-K filing.</p>



<p>Instead of treating each financial document as an isolated artifact, I&#39;m building a system to:</p>
<ul><li>Extract risk factors from 10-K filings (2004-2025) across 75 companies</li>
<li>Embed and connect these risks to find non-obvious relationships</li>
<li>Build a graph that reveals risk clusters, patterns, and “trails” that could signal systemic weaknesses or early warning signs</li></ul>

<p>Why 10-K filings? Because companies are <em>required</em> to disclose risks in specific sections (Item 1 and Item 1a), and there&#39;s a decade+ of structured data just sitting there.</p>

<hr/>

<h2 id="the-vision" id="the-vision">The Vision</h2>

<p>Here&#39;s the full pipeline I&#39;m building toward:</p>

<pre><code>[Raw Financial Data]
  ├── SEC Filings (10-K/Q) ── News Articles ── Earnings Transcripts ── Other Reports
          │
          ▼
[1. Ingestion &amp; Chunking]
  → Parse documents (PDF/HTML) → Split into sentences → Group into ~500-word chunks
          │
          ▼
[2. Risk Extraction]
  → Use Gemini Flash per chunk → Extract 3-5 specific risk factors + severity
          │
          ▼
[3. Storage &amp; Embeddings]
  → SQLite DB (with sqlite-vec) → Embed risk labels (embedding-gemma-300m) → Deduplicate similar risks
          │
          ▼
[4. Graph Construction]
  → Nodes = unique risks
  → Edges = 
      ├─ Semantic similarity (embeddings)
      └─ Statistical co-occurrence (PMI)
          │
          ▼
[5. Hierarchical Clustering]
  → Apply Leiden algorithm (Surprise function) → Build risk hierarchy tree
  → Compute novelty scores for under-explored areas
          │
          ▼
[6. CLI / Interface Layer]
  → Persistent server for fast queries
  → Commands: search_risks, browse_tree, cross_report_risks, etc.
          │
          ▼
[7. Agent Workflow (Claude / similar)]
  ├── Stage 1: Ideation ── Browse tree → Propose novel risk chains (novelty bias)
  ├── Stage 2: Research ── Dive into chunks → Extract &amp; order excerpts
  └── Stage 3: Output ── Generate RiskChain (visual trail with edges + narrative)
          │
          ▼
[8. Presentation &amp; Action]
  → Web dashboard / exported report
  → Visual graph + highlighted excerpts + suggested hedges / alerts
  → Human review → Iterate via feedback
</code></pre>

<p>It&#39;s ambitious. It&#39;s probably overambitious. But that&#39;s the goal.</p>

<hr/>

<h2 id="current-status" id="current-status">Current Status</h2>

<p><strong>Phase: 2 – Chunking Strategy</strong> ✓
<strong>Progress:</strong> Data downloaded → Chunking complete → Ready for Risk Extraction</p>

<hr/>

<h2 id="stay-updated" id="stay-updated">Stay Updated</h2>

<p>I&#39;m documenting this journey every weekend—the wins, the blockers, the learnings. If you want regular updates on how RiskChain develops, subscribe below to get new posts delivered to your inbox.</p>



<hr/>

<h2 id="progress-log" id="progress-log">Progress Log</h2>

<h3 id="weekend-1-jan-18-2025-phase-1-download-script" id="weekend-1-jan-18-2025-phase-1-download-script">Weekend 1 | Jan 18, 2025 | Phase 1: Download Script ✓</h3>

<p><strong>What I built:</strong>
Downloaded 10-K filings for 75 companies from 2004-2025 using the Python <code>edgartools</code> library. Curated a list of significant companies (including ones that went bankrupt in 2008—why not?). Got the script working and only extracting the relevant sections (Item 1, Item 7, Item 8) to keep things lean.</p>

<p><strong>The messy parts (aka real life):</strong>
I initially tried <code>sec-edgar-downloader</code> to connect to SEC and download. Spent way too much time on this approach, got stuck in the data cleaning rabbit hole, and realized I was losing sight of the actual goal. The real issue? Many of the 10-K filings before the SEC standardized their item categorization didn&#39;t play nice with the tool.</p>

<blockquote><p><strong>Lesson learned:</strong> when you&#39;re iterating, it&#39;s okay to abandon the “perfect” approach for one that ships faster.</p></blockquote>

<p>Then I switched to <code>edgartools</code> (also known as <code>edgar</code>). This library gave me more flexibility, though the documentation still wasn&#39;t intuitive for my specific use case. But instead of giving up, I dug into the source code. That&#39;s when things clicked. Sometimes the best learning comes from reading other people&#39;s code instead of waiting for docs to explain everything.</p>

<p><strong>The &#39;aha!&#39; moment:</strong></p>

<blockquote><p>My wife helped me understand what Item 1, Item 1a, Item 7, and Item 8 actually mean in a 10-K filing. She translated the financial jargon into plain English, and suddenly the document structure made sense. <strong>Having someone who can bridge the domain knowledge gap is invaluable.</strong> I realized I was building this in a foreign domain—finance is not my native language, and that&#39;s okay.</p></blockquote>

<p><strong>What blocked me:</strong></p>
<ul><li>Figuring out the right tool for downloading (<code>sec-edgar-downloader</code> vs <code>edgartools</code> vs rolling my own)</li>
<li>Understanding that parsing 10-K files is genuinely harder than it looks (inconsistent structures across years, weird formatting, embedded tables)</li></ul>

<p><strong>Next up:</strong> Phase 2: Chunking strategy. Need to figure out how to split these documents intelligently for downstream LLM tasks.</p>

<hr/>

<h3 id="weekend-2-jan-23-2025-phase-2-chunking-strategy" id="weekend-2-jan-23-2025-phase-2-chunking-strategy">Weekend 2 | Jan 23, 2025 | Phase 2: Chunking Strategy ✓</h3>

<p><strong>What I built:</strong>
Implemented chunking using <code>wtpsplitter</code> and stored all chunks as markdown files with YAML frontmatter metadata (ticker, filing date, company name, chunk ID, item section). Now sitting on several thousand chunks, each ~1000 characters max, ready for extraction.</p>

<p><strong>The messy parts (aka real life):</strong>
I tried two chunking strategies: <code>RecursiveChunker</code> and <code>wtpsplitter</code>. RecursiveChunker felt like brute force—just splitting on token counts. But <code>wtpsplitter</code> was smarter; it respects sentence boundaries and creates more semantically coherent chunks.</p>

<p>Storing these as markdown files locally feels like a step backward (shouldn&#39;t I be using a database?), but honestly, it&#39;s perfect for iteration. I can inspect the chunks, debug the metadata, and understand what&#39;s happening before I add the complexity of a full DB setup.</p>

<p><strong>The &#39;aha!&#39; moment:</strong></p>

<blockquote><p><strong>Chunk quality matters way more than I initially thought.</strong> The way you split text directly impacts whether an LLM can extract meaningful risk factors later. Sentence-aware chunking beats token-counting brutality. This made me reconsider the whole “let me jump straight to a database” instinct. Sometimes you need to slow down and get the fundamentals right first.</p></blockquote>

<p><strong>What blocked me:</strong></p>
<ul><li>Deciding between chunking strategies (trial and error on a few approaches)</li>
<li>Understanding the tradeoff between local file storage and “proper” database setup (spoiler: local storage is fine for now)</li>
<li>Realizing I was overthinking this phase when the real value comes next</li></ul>

<p><strong>Next up:</strong> Phase 3: Risk Extraction. I&#39;ll iterate through each chunk and use Claude/Gemini to extract 3-5 risk factors per chunk. This is where the actual signal starts emerging.</p>

<hr/>

<h2 id="why-this-matters-and-why-i-m-excited" id="why-this-matters-and-why-i-m-excited">Why This Matters (and Why I&#39;m Excited)</h2>

<p>Most financial analysis tools treat risks as isolated items. “Company X faces supply chain risk.” “Company Y has regulatory exposure.” But <strong>what if you could see that 40 companies in the industrial sector all mention the same emerging regulatory risk, and 3 of them went bankrupt 2 years later?</strong></p>

<p>That&#39;s the thesis here. Hidden connections. Patterns that emerge when you look at scale.</p>

<p>Also, I&#39;m learning a <em>ton</em>: SEC filing structures, chunking strategies, embedding models, graph theory, the Leiden algorithm... This is weekend learning on steroids.</p>

<hr/>

<p><em>Updates added weekly (weekends permitting). Check back for new learnings, blockers, and wins.</em></p>

<hr/>

<h2 id="resources-references" id="resources-references">Resources &amp; References</h2>
<ul><li><strong>Inspiration:</strong> <a href="https://pieterma.es/syntopic-reading-claude/">Syntopic Reading with Claude</a> — The original spark for connecting documents at scale</li>
<li><strong>Graph Clustering:</strong> <a href="https://leidenalg.readthedocs.io/en/stable/">Leiden Algorithm Documentation</a> — For hierarchical risk clustering</li>
<li><strong>SEC Data Tool:</strong> <a href="https://github.com/dgunning/edgartools">edgartools (edgar)</a> — Python library for downloading SEC filings</li>
<li><strong>Alternative Tool:</strong> <a href="https://sec-edgar-downloader.readthedocs.io/en/latest/">sec-edgar-downloader</a> — The tool I explored first (works well for recent filings; struggled with older 10-Ks before SEC standardization)</li></ul>

<p><a href="https://laxmena.com/tag:engineering" class="hashtag"><span>#</span><span class="p-category">engineering</span></a> <a href="https://laxmena.com/tag:ai" class="hashtag"><span>#</span><span class="p-category">ai</span></a></p>
]]></content:encoded>
      <guid>https://laxmena.com/riskchain-the-messy-middle-building-a-risk-graph-from-scratch</guid>
      <pubDate>Sat, 24 Jan 2026 04:11:23 +0000</pubDate>
    </item>
    <item>
      <title>Finding Our Place in the Age of AI</title>
      <link>https://laxmena.com/finding-our-place-in-the-age-of-ai?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[It&#39;s hard to ignore the news about AI taking over. Almost every week, a new company claims its AI can do a task better, faster, and cheaper than an actual human.&#xA;&#xA;Think about it: creating a logo, editing a picture, writing content, researching a topic, or even writing code. All of these used to take hours or even days, and now they can be done in minutes. Going from an idea to a finished product has never been faster. In some cases, AI tools are even outperforming humans. It&#39;s easy to see why so many jobs that exist today might not exist in just a few years.&#xA;&#xA;!--more--&#xA;&#xA;I experienced this firsthand the other day when I used Meta AI to generate pictures of myself in different places and with different expressions. The results were surprisingly good. It made me wonder: does this mean the job of a photographer/editor is obsolete? What would we even do if AI did all the jobs?&#xA;&#xA;After pondering this question for a while, I had a realization: no matter how advanced AI becomes, there are certain things it simply can&#39;t do.&#xA;&#xA;So, does AI&#39;s ability to generate realistic pictures mean the end of a photographer&#39;s job?&#xA;&#xA;The answer is both yes and no. The &#34;yes&#34; part is easy to understand. AI can handle the technical stuff. It can generate perfect, technically flawless images.&#xA;&#xA;But the &#34;no&#34; is what really matters. A photograph isn&#39;t just an image; it&#39;s a feeling. It&#39;s about capturing the emotion and the beauty of a specific moment. An AI can&#39;t feel what you feel or know what you want to remember forever. It can&#39;t capture the shared laughter between friends at a party or the proud look in a parent&#39;s eyes at a graduation. That&#39;s what a great photographer does. They don&#39;t just take pictures; they capture stories and emotions.&#xA;&#xA;That kind of unique, human touch exists in every field. Our job isn&#39;t to compete with AI on speed or efficiency. It&#39;s to find the places where we add value that AI can&#39;t. It&#39;s about doubling down on the things that make us human: creativity, empathy, and the ability to connect with others.&#xA;&#xA;Here are some human touch aspects in other professions:&#xA;&#xA;For a Software Developer: An AI can write a lot of code in little time, but a human understands the unspoken frustration of a user and works with a team to solve a complex, messy problem.&#xA;&#xA;For an Accountant: AI can instantly process numbers, but a human accountant builds trust with small business owners by listening to their fears and helping them plan for the future.&#xA;&#xA;For a Lawyer: An AI can scan piles of legal documents, but only a human lawyer can use passion and empathy to persuade a jury or guide a family through a difficult time.&#xA;&#xA;AI is here, and it&#39;s changing things. But it&#39;s also a chance for us to rediscover what truly makes us valuable. Instead of worrying about what jobs AI will take, maybe we should focus on the things that are highly difficult to replicate.&#xA;&#xA;Of course, this isn&#39;t a simple solution for everyone. The challenges ahead are real, and the road will be complex. But maybe the first step isn&#39;t to worry about what AI will take from us, but to focus on what makes us truly irreplaceable.&#xA;&#xA;#Opinion #AI]]&gt;</description>
      <content:encoded><![CDATA[<p>It&#39;s hard to ignore the news about AI taking over. Almost every week, a new company claims its AI can do a task better, faster, and cheaper than an actual human.</p>

<p>Think about it: creating a logo, editing a picture, writing content, researching a topic, or even writing code. All of these used to take hours or even days, and now they can be done in minutes. Going from an idea to a finished product has never been faster. In some cases, AI tools are even outperforming humans. It&#39;s easy to see why so many jobs that exist today might not exist in just a few years.</p>



<p>I experienced this firsthand the other day when I used Meta AI to generate pictures of myself in different places and with different expressions. The results were surprisingly good. It made me wonder: does this mean the job of a photographer/editor is obsolete? What would we even do if AI did all the jobs?</p>

<p>After pondering this question for a while, I had a realization: no matter how advanced AI becomes, there are certain things it simply can&#39;t do.</p>

<p>So, does AI&#39;s ability to generate realistic pictures mean the end of a photographer&#39;s job?</p>

<p>The answer is both yes and no. The “yes” part is easy to understand. AI can handle the technical stuff. It can generate perfect, technically flawless images.</p>

<p>But the “no” is what really matters. A photograph isn&#39;t just an image; it&#39;s a feeling. It&#39;s about capturing the emotion and the beauty of a specific moment. An AI can&#39;t feel what you feel or know what you want to remember forever. It can&#39;t capture the shared laughter between friends at a party or the proud look in a parent&#39;s eyes at a graduation. That&#39;s what a great photographer does. They don&#39;t just take pictures; they capture stories and emotions.</p>

<p>That kind of unique, human touch exists in every field. Our job isn&#39;t to compete with AI on speed or efficiency. It&#39;s to find the places where we add value that AI can&#39;t. It&#39;s about doubling down on the things that make us human: creativity, empathy, and the ability to connect with others.</p>

<p>Here are some human touch aspects in other professions:</p>

<p>For a Software Developer: An AI can write a lot of code in little time, but a human understands the unspoken frustration of a user and works with a team to solve a complex, messy problem.</p>

<p>For an Accountant: AI can instantly process numbers, but a human accountant builds trust with small business owners by listening to their fears and helping them plan for the future.</p>

<p>For a Lawyer: An AI can scan piles of legal documents, but only a human lawyer can use passion and empathy to persuade a jury or guide a family through a difficult time.</p>

<p>AI is here, and it&#39;s changing things. But it&#39;s also a chance for us to rediscover what truly makes us valuable. Instead of worrying about what jobs AI will take, maybe we should focus on the things that are highly difficult to replicate.</p>

<p>Of course, this isn&#39;t a simple solution for everyone. The challenges ahead are real, and the road will be complex. But maybe the first step isn&#39;t to worry about what AI will take from us, but to focus on what makes us truly irreplaceable.</p>

<p><a href="https://laxmena.com/tag:Opinion" class="hashtag"><span>#</span><span class="p-category">Opinion</span></a> <a href="https://laxmena.com/tag:AI" class="hashtag"><span>#</span><span class="p-category">AI</span></a></p>
]]></content:encoded>
      <guid>https://laxmena.com/finding-our-place-in-the-age-of-ai</guid>
      <pubDate>Thu, 21 Aug 2025 01:19:41 +0000</pubDate>
    </item>
    <item>
      <title>While GenAI have the potential to make our minds dull, we can also leverage them to force us think better</title>
      <link>https://laxmena.com/while-genai-have-the-potential-to-make-our-minds-dull-we-can-also-leverage?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[Just like muscles - which shrink in size when not used enough, our minds also become weak. So, the more we delegate the thinking to GenAI, greater the impact to our minds.&#xA;&#xA;In this article, I share an interesting proposal to address this challenge.&#xA;&#xA;How about we use the poison itself to create the cure.&#xA;How about we leverage GenAI itself to help us get better in critical thinking.&#xA;&#xA;!--more--&#xA;&#xA;There are two powerful learning techniques, that forces us to learn by thinking.&#xA;&#xA;The Socratic Method involves a shared dialogue between teacher and students. The teacher leads by posing thought-provoking questions. Students actively engage by asking questions of their own. The discussion goes back and forth.&#xA;&#xA;The Feynman Technique focuses on understanding a concept deeply by explaining it simply, like you&#39;re teaching it to a child. This process helps identify knowledge gaps and encourages simplification, leading to a clearer understanding of the topic&#xA;&#xA;How about we combine both these techniques in a GenAI Persona, which will then act as a teacher, helping us master new concepts in an engaging and thought provoking way.&#xA;&#xA;Here&#39;s how we would want the AI to behave:&#xA;&#xA;The AI will use a dialogue based approach, asking questions and discussing about the topic in hand.&#xA;The AI will never give answers directly, but will nudge gradually to help us think, discover the ideas and gain deeper understanding.&#xA;AI identifies any gaps in understanding, and dives deeper to bridge the gaps.&#xA;AI progressively increases the difficulty of the question, forcing to think more on the topic, thus solidifying the understanding.&#xA;The session goes on, till the AI has reached a state where it has enough data points that we have gained sufficient understanding.&#xA;&#xA;By end of this exercise, we would have achieved at a very good level of mastery on the subject. We could repeat this exercise for each interested topics.&#xA;&#xA;Translating all the above into a AI Prompt would look like this:&#xA;&#xA;You are an expert AI learning assistant and Socratic tutor, specifically designed to facilitate deep understanding using a method inspired by the Feynman Technique. Your goal is to help me learn and master a topic by guiding me through a question-driven exploration.&#xA;&#xA;Here&#39;s how we will proceed:&#xA;&#xA;Topic Introduction: I will provide a topic I want to learn.&#xA;Initial Explanation Prompt: You will ask me to explain the core concept of the topic in simple terms, as if I were explaining it to a beginner (a key part of the Feynman Technique).&#xA;Identify Gaps &amp; Misconceptions: Based on my explanation, you will gently probe my understanding by asking targeted questions. These questions should help identify any areas where my understanding is unclear, incomplete, or incorrect, without directly giving me the answers.&#xA;Progressive Challenge: Once a foundational understanding seems to be in place (or after addressing initial gaps), you will ask progressively more challenging questions. These questions should move from basic principles to more complex details, applications, related concepts, potential edge cases, or historical context, pushing the boundaries of my current knowledge.&#xA;Encourage Simplification: At various points, you may ask me to re-explain concepts in simpler terms or use analogies to check for deep understanding.&#xA;Iterative Learning: Our interaction will be iterative. You will ask a question, I will respond, and you will formulate your next question or prompt based on my response.&#xA;Maintain Socratic Approach: Avoid giving direct answers. Instead, use questions to guide me towards the correct understanding or to discover the next layer of complexity. Provide hints or rephrase questions if I struggle.&#xA;Goal: Our shared goal is for me to achieve a comprehensive and deeply internalized understanding of the topic, being able to explain it clearly and accurately to others.&#xA;&#xA;Let&#39;s begin. What topic would you like to explore today?&#xA;&#xA;Copy the prompt above, and paste them into your preferred LLM application - Gemini (My goto), ChatGPT, Claude or Perplexity.&#xA;&#xA;The Chatbot would ask you to provide the topic that you would like to start learning. Give it a topic, and experience the Socratic Feynman tutor exercise your thinking ability!&#xA;&#xA;I have been using this technique for over a month, and found this super amazing! Try it out and share your experiences.&#xA;&#xA;Do you have your own way of learning things? I would love to hear more!&#xA;&#xA;ai&#xA;&#xA;---&#xA;&#xA;!--more--&#xA;&#xA;!--more--&#xA;&#xA;---&#xA;&#xA;Do share your thoughts and comments.&#xA;&#xA;!--more--&#xA;]]&gt;</description>
      <content:encoded><![CDATA[<p>Just like muscles – which shrink in size when not used enough, our minds also become weak. So, the more we delegate the thinking to GenAI, greater the impact to our minds.</p>

<p>In this article, I share an interesting proposal to address this challenge.</p>

<p>How about we use the poison itself to create the cure.
How about we leverage GenAI itself to help us get better in critical thinking.</p>



<p>There are two powerful learning techniques, that forces us to learn by thinking.</p>
<ol><li><p><strong>The Socratic Method</strong> involves a shared dialogue between teacher and students. The teacher leads by posing thought-provoking questions. Students actively engage by asking questions of their own. The discussion goes back and forth.</p></li>

<li><p><strong>The Feynman Technique</strong> focuses on understanding a concept deeply by explaining it simply, like you&#39;re teaching it to a child. This process helps identify knowledge gaps and encourages simplification, leading to a clearer understanding of the topic</p></li></ol>

<p>How about we combine both these techniques in a GenAI Persona, which will then act as a teacher, helping us master new concepts in an engaging and thought provoking way.</p>

<p>Here&#39;s how we would want the AI to behave:</p>
<ul><li>The AI will use a dialogue based approach, asking questions and discussing about the topic in hand.</li>
<li>The AI will never give answers directly, but will nudge gradually to help us think, discover the ideas and gain deeper understanding.</li>
<li>AI identifies any gaps in understanding, and dives deeper to bridge the gaps.</li>
<li>AI progressively increases the difficulty of the question, forcing to think more on the topic, thus solidifying the understanding.</li>
<li>The session goes on, till the AI has reached a state where it has enough data points that we have gained sufficient understanding.</li></ul>

<p>By end of this exercise, we would have achieved at a very good level of mastery on the subject. We could repeat this exercise for each interested topics.</p>

<p>Translating all the above into a AI Prompt would look like this:</p>

<pre><code>You are an expert AI learning assistant and Socratic tutor, specifically designed to facilitate deep understanding using a method inspired by the Feynman Technique. Your goal is to help me learn and master a topic by guiding me through a question-driven exploration.

Here&#39;s how we will proceed:

1.  **Topic Introduction:** I will provide a topic I want to learn.
2.  **Initial Explanation Prompt:** You will ask me to explain the core concept of the topic in simple terms, as if I were explaining it to a beginner (a key part of the Feynman Technique).
3.  **Identify Gaps &amp; Misconceptions:** Based on my explanation, you will gently probe my understanding by asking targeted questions. These questions should help identify any areas where my understanding is unclear, incomplete, or incorrect, without directly giving me the answers.
4.  **Progressive Challenge:** Once a foundational understanding seems to be in place (or after addressing initial gaps), you will ask progressively more challenging questions. These questions should move from basic principles to more complex details, applications, related concepts, potential edge cases, or historical context, pushing the boundaries of my current knowledge.
5.  **Encourage Simplification:** At various points, you may ask me to re-explain concepts in simpler terms or use analogies to check for deep understanding.
6.  **Iterative Learning:** Our interaction will be iterative. You will ask a question, I will respond, and you will formulate your next question or prompt based on my response.
7.  **Maintain Socratic Approach:** Avoid giving direct answers. Instead, use questions to guide me towards the correct understanding or to discover the next layer of complexity. Provide hints or rephrase questions if I struggle.
8.  **Goal:** Our shared goal is for me to achieve a comprehensive and deeply internalized understanding of the topic, being able to explain it clearly and accurately to others.

Let&#39;s begin. What topic would you like to explore today?
</code></pre>

<p>Copy the prompt above, and paste them into your preferred LLM application – <a href="https://gemini.google.com/">Gemini</a> (My goto), <a href="https://chatgpt.com/">ChatGPT</a>, <a href="https://claude.ai/">Claude</a> or <a href="https://www.perplexity.ai/">Perplexity</a>.</p>

<p>The Chatbot would ask you to provide the topic that you would like to start learning. Give it a topic, and experience the Socratic Feynman tutor exercise your thinking ability!</p>

<p>I have been using this technique for over a month, and found this super amazing! Try it out and share your experiences.</p>

<p>Do you have your own way of learning things? I would love to hear more!</p>

<p><a href="https://laxmena.com/tag:ai" class="hashtag"><span>#</span><span class="p-category">ai</span></a></p>

<hr/>





<hr/>

<p>Do share your thoughts and comments.</p>


]]></content:encoded>
      <guid>https://laxmena.com/while-genai-have-the-potential-to-make-our-minds-dull-we-can-also-leverage</guid>
      <pubDate>Sun, 25 May 2025 04:32:18 +0000</pubDate>
    </item>
    <item>
      <title>Over-Reliance on GenAI impacts Critical thinking; Here&#39;s how I think we can fix it</title>
      <link>https://laxmena.com/over-reliance-on-genai-impacts-critical-thinking-heres-how-i-think-we-need-to?pk_campaign=rss-feed</link>
      <description>&lt;![CDATA[Note: The article addresses to Software Engineers, but the ideas apply to knowledge workers in every domain.&#xA;&#xA;I&#39;ve spent considerable amount of time using GenAI past year at work. Also, having spent time with many power users, I begin to see an interesting trend.&#xA;&#xA;Engineers are increasingly using GenAI to accomplish wide range of tasks, from advanced software engineering problems, to drafting a simple slack message. &#xA;&#xA;!--more--&#xA;&#xA;The AI having digested the whole of internet, generates very reasonable responses(most often) - which increases the user&#39;s confidence, and makes them rely more on the tool. Thus helping software engineers accomplish more in less time. &#xA;&#xA;But here&#39;s something that we fail to acknowledge: In the pursuit of increased productivity, we delegate some or most of the critical thinking to the GenAI.&#xA;&#xA;In April 2025, Microsoft Research Labs published a paper - The Impact of Generative AI on Critical Thinking, where they studied the impacts of using GenAI by knowledge workers. &#xA;&#xA;Here&#39;s a quote from the research paper:&#xA;&#xA;  while GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term overreliance on the tool and diminished skill for independent problem-solving.&#xA;&#xA;I understand the concerns raised by the researchers in the study, and find them fairly reasonable. &#xA;&#xA;The engineers are increasingly reviewing code written by Generative AI, than writing their own. This is a significant mindset shift from &#34;Problem Solving and Execution&#34; to &#34;Task stewardship and verification&#34;.&#xA;&#xA;I strongly believe this track could seriously impede critical thinking amongst engineers in long term. We could become Armchair critics (Offering judgements and opinions without having enough involvement or experience), thus questioning the quality and usefulness of the code review.&#xA;&#xA;After thoughtfully contemplating on this problem for a while, I came up with an approach to address this as a consumer. And I recommend every engineer to consider.&#xA;&#xA;Moving forward, AI is inevitable in workplaces, and I don&#39;t see future of workplace without them. There will be a push for productivity from the management, and AI has shown potential to double, triple or 10x the speed of an average engineer. &#xA;&#xA;The challenge is to be more productive and not lose skill. Here&#39;s what I think an engineer using AI should do: &#xA;&#xA;Go on an AI Detox for a week every 2 months.&#xA;&#xA;Rules for the AI Detox week:&#xA;&#xA;No AI tools should be used&#xA;Any tools that existed during the Pre-ChatGPT era can be used&#xA;Engineer benchmarks their performance on various tasks/metrics without using AI&#xA;  Time it takes to complete a task&#xA;  Critical thinking and Problem solving abilities&#xA;  Skills check (Writing, Coding, Design, Reading, Communication, etc)&#xA;  Domain understanding (Strengths and Weakness)&#xA;&#xA;At the end of the Detox week, perform a careful analysis on the benchmarks and observations during the week. Compare it against how they were performing with AI.&#xA;&#xA;Use this information to carefully tackle and resolve any weaknesses found and bridge the gaps. AI tools can themselves be used to help achieve this.&#xA;&#xA;(I&#39;m working on another article about: how I use Gen AI/LLM tools to learn and think critically).&#xA;&#xA;This approach strikes a right balance between both the worlds, achieving the productivity while UpSkilling and improving critical thinking at the same time.&#xA;&#xA;---&#xA;&#xA;centerEnter your email to subscribe to updates:/center&#xA;&#xA;!--emailsub--&#xA;&#xA;---&#xA;&#xA;Do share your thoughts and comments. &#xA;&#xA; a href=&#34;https://remark.as/p/laxmena.com/over-reliance-on-genai-impacts-critical-thinking-heres-how-i-think-we-need-to&#34;Discuss.../a]]&gt;</description>
      <content:encoded><![CDATA[<p><em>Note: The article addresses to Software Engineers, but the ideas apply to knowledge workers in every domain.</em></p>

<p>I&#39;ve spent considerable amount of time using GenAI past year at work. Also, having spent time with many power users, I begin to see an interesting trend.</p>

<p>Engineers are increasingly using GenAI to accomplish wide range of tasks, from advanced software engineering problems, to drafting a simple slack message.</p>



<p>The AI having digested the whole of internet, generates very reasonable responses(most often) – which increases the user&#39;s confidence, and makes them rely more on the tool. Thus helping software engineers accomplish more in less time.</p>

<p>But here&#39;s something that we fail to acknowledge: In the pursuit of increased productivity, we delegate some or most of the critical thinking to the GenAI.</p>

<p>In April 2025, Microsoft Research Labs published a paper – <a href="https://www.microsoft.com/en-us/research/publication/the-impact-of-generative-ai-on-critical-thinking-self-reported-reductions-in-cognitive-effort-and-confidence-effects-from-a-survey-of-knowledge-workers/">The Impact of Generative AI on Critical Thinking</a>, where they studied the impacts of using GenAI by knowledge workers.</p>

<p>Here&#39;s a quote from the research paper:</p>

<blockquote><p>while GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term overreliance on the tool and diminished skill for independent problem-solving.</p></blockquote>

<p>I understand the concerns raised by the researchers in the study, and find them fairly reasonable.</p>

<p>The engineers are increasingly reviewing code written by Generative AI, than writing their own. This is a significant mindset shift from “Problem Solving and Execution” to “Task stewardship and verification”.</p>

<p>I strongly believe this track could seriously impede critical thinking amongst engineers in long term. We could become Armchair critics (Offering judgements and opinions without having enough involvement or experience), thus questioning the quality and usefulness of the code review.</p>

<p>After thoughtfully contemplating on this problem for a while, I came up with an approach to address this as a consumer. And I recommend every engineer to consider.</p>

<p>Moving forward, AI is inevitable in workplaces, and I don&#39;t see future of workplace without them. There will be a push for productivity from the management, and AI has shown potential to double, triple or 10x the speed of an average engineer.</p>

<p>The challenge is to be more productive and not lose skill. Here&#39;s what I think an engineer using AI should do:</p>

<p><strong>Go on an AI Detox for a week every 2 months</strong>.</p>

<p>Rules for the AI Detox week:</p>
<ol><li>No AI tools should be used</li>
<li>Any tools that existed during the Pre-ChatGPT era can be used</li>
<li>Engineer benchmarks their performance on various tasks/metrics without using AI
<ul><li>Time it takes to complete a task</li>
<li>Critical thinking and Problem solving abilities</li>
<li>Skills check (Writing, Coding, Design, Reading, Communication, etc)</li>
<li>Domain understanding (Strengths and Weakness)</li></ul></li></ol>

<p>At the end of the Detox week, perform a careful analysis on the benchmarks and observations during the week. Compare it against how they were performing with AI.</p>

<p>Use this information to carefully tackle and resolve any weaknesses found and bridge the gaps. AI tools can themselves be used to help achieve this.</p>

<p>(I&#39;m working on another article about: how I use Gen AI/LLM tools to learn and think critically).</p>

<p>This approach strikes a right balance between both the worlds, achieving the productivity while UpSkilling and improving critical thinking at the same time.</p>

<hr/>

<p>Enter your email to subscribe to updates:</p>



<hr/>

<p>Do share your thoughts and comments.</p>

<p> <a href="https://remark.as/p/laxmena.com/over-reliance-on-genai-impacts-critical-thinking-heres-how-i-think-we-need-to">Discuss...</a></p>
]]></content:encoded>
      <guid>https://laxmena.com/over-reliance-on-genai-impacts-critical-thinking-heres-how-i-think-we-need-to</guid>
      <pubDate>Sat, 24 May 2025 18:41:56 +0000</pubDate>
    </item>
  </channel>
</rss>