{"id":77,"date":"2023-12-14T17:53:00","date_gmt":"2023-12-14T17:53:00","guid":{"rendered":"https:\/\/balamurali.in\/blog\/?p=77"},"modified":"2026-02-23T14:27:38","modified_gmt":"2026-02-23T14:27:38","slug":"building-a-rule-based-engine-to-extract-founding-years-from-html","status":"publish","type":"post","link":"https:\/\/balamurali.in\/blog\/tech-posts\/building-a-rule-based-engine-to-extract-founding-years-from-html\/","title":{"rendered":"Building a Rule-Based Engine to Extract Founding Years from HTML"},"content":{"rendered":"\n

In the world of Private Equity, Vintage<\/strong> is everything.<\/p>\n\n\n\n

An investor looks at a company with $1M in revenue and asks: “Did they reach this in 12 months, or 12 years?”<\/em> The answer determines if the startup is a rocket ship or a zombie.<\/p>\n\n\n\n

At Tracxn, we needed the Founding Year<\/strong> for millions of companies to calculate this vintage.<\/p>\n\n\n\n

The obvious solution was to scrape LinkedIn. But LinkedIn is a fortress\u2014expensive to proxy, guarded by anti-bot walls, and full of user-generated noise (employees guessing start dates).<\/p>\n\n\n\n

We realized the best source of truth wasn’t a third-party aggregator; it was the company’s own diary. Every startup loves to tell its story on its website. We just needed a way to read it.<\/p>\n\n\n\n

Instead of using expensive NLP models to “read” history, I built a script using Pure Python Heuristics<\/strong>. Here is the detailed breakdown of how we architected this system without a single GPU.<\/p>\n\n\n\n

The Core Philosophy: “The Source of Truth is the Site”<\/h2>\n\n\n\n

We operated on a simple assumption: If a company exists, they have an “About” page.<\/strong> And on that page, they will inevitably mention when they started.<\/p>\n\n\n\n

Our goal wasn’t to parse the whole internet. It was to find that one specific sentence\u2014“Established in 2015”<\/em>\u2014and extract the integer.<\/p>\n\n\n\n

Step 1: The Navigator (Smart Page Discovery)<\/h2>\n\n\n\n

Finding the founding year is harder than finding a legal name because it rarely lives on the homepage. You have to dig.<\/p>\n\n\n\n

We couldn’t crawl every page (too slow), so we built a Priority Queue Navigator<\/strong>.<\/p>\n\n\n\n

    \n
  1. Extract:<\/strong> Grab all hyperlinks (<a href><\/code>) from the homepage.<\/li>\n\n\n\n
  2. Filter & Rank:<\/strong> We scored links based on keywords in the URL or anchor text.\n
      \n
    • Tier 1 (High Probability):<\/strong> history<\/code>, journey<\/code>, timeline<\/code>, milestones<\/code>, legacy<\/code>.<\/li>\n\n\n\n
    • Tier 2 (Medium Probability):<\/strong> about<\/code>, story<\/code>, who-we-are<\/code>.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • Visit:<\/strong> The script would visit the highest-scoring page first. If it found “Our Journey,” it ignored “Contact Us.”<\/li>\n<\/ol>\n\n\n\n

      This targeted approach meant we usually found the data in just one HTTP request<\/strong> after the homepage.<\/p>\n\n\n\n

      Step 2: The Scanner (The Year Regex)<\/h2>\n\n\n\n

      Once we landed on the target page (e.g., swiggy.com\/about-us<\/code>), we had to find the year. But searching for 4-digit numbers is dangerous. You\u2019ll catch zip codes, prices ($2000), and support hotlines.<\/p>\n\n\n\n

      We used a strict Year Regex<\/strong> pattern:
      r'\\b(19|20)\\d{2}\\b'<\/code><\/p>\n\n\n\n

      This acted as our first filter:<\/p>\n\n\n\n

        \n
      • Accept:<\/strong> 1998, 2015, 2023.<\/li>\n\n\n\n
      • Reject:<\/strong> 5000 (too high), 1000 (too low).<\/li>\n<\/ul>\n\n\n\n

        We also applied a Noise Filter<\/strong> to remove common corporate red herrings:<\/p>\n\n\n\n

          \n
        • Current Year:<\/strong> If extracted year == datetime.now().year<\/code>, it\u2019s likely a copyright date. Discard.<\/strong><\/li>\n\n\n\n
        • ISO Standards:<\/strong> “ISO 9001” or “ISO 27001” are certification numbers, not dates. Discard.<\/strong><\/li>\n<\/ul>\n\n\n\n

          Step 3: The Context Engine (The “Sliding Window”)<\/h2>\n\n\n\n

          This was the “No AI” magic. Finding “2014” isn’t enough. The text could say “We won an award in 2014”<\/em> or “Our founder graduated in 2014.”<\/em><\/p>\n\n\n\n

          We needed to link the Year<\/strong> to the concept of Creation<\/strong>.<\/p>\n\n\n\n

          Instead of training a neural network to understand sentence structure, we used a Sliding Window Algorithm<\/strong>.<\/p>\n\n\n\n

            \n
          1. Identify Keywords:<\/strong> We compiled a dictionary of “Birth Words”: ['founded', 'established', 'estd', 'incorporated', 'started', 'launched', 'inception', 'born']<\/code>.<\/li>\n\n\n\n
          2. Measure Distance:<\/strong> For every candidate year found, we looked at the 10 words before and after it.<\/li>\n\n\n\n
          3. Score:<\/strong>\n
              \n
            • “Founded<\/strong> in 2015<\/strong>“<\/em> -> Distance: 1 word. Strong Match.<\/strong><\/li>\n\n\n\n
            • “Established<\/strong> leaders\u2026 won an award in 2015<\/strong>“<\/em> -> Distance: 6 words + punctuation barrier. Weak Match.<\/strong><\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n

              This simple proximity check filtered out 90% of false positives without parsing a single grammar tree.<\/p>\n\n\n\n

              Step 4: The “Min-Year” Heuristic<\/h2>\n\n\n\n

              Startup “About” pages are often confusing.<\/p>\n\n\n\n

              \n

              “Founded in 2010, we launched our app in 2012 and went global in 2015.”<\/em><\/p>\n<\/blockquote>\n\n\n\n

              Our script would extract three valid years: 2010, 2012, 2015. Which one is the “Founding Year”?<\/p>\n\n\n\n

              We applied the Min-Year Heuristic<\/strong>: In the context of a company history page, the earliest<\/em> valid date associated with a “Birth Word” is almost always the founding date.<\/p>\n\n\n\n

              Step 5: Verification (The “Why”)<\/h2>\n\n\n\n

              Data without proof is useless. If our database said “Founded in 2011,” an analyst might doubt it.<\/p>\n\n\n\n

              Unlike a black-box AI model, our script returned a structured evidence object:<\/p>\n\n\n\n

              {\n  \"founding_year\": 2011,\n  \"confidence\": \"High\",\n  \"source_url\": \"https:\/\/company.com\/our-story\",\n  \"context_snippet\": \"...our journey began in 2011 when...\"\n}<\/code><\/pre>\n\n\n\n
              This allowed human verifiers to click a link and confirm the data in seconds, rather than searching the site from scratch.<\/p>\n\n\n\n
              Summary<\/h2>\n\n\n\n
              The goal of this project was to move from “Identity” (Who are they?) to “History” (When did they start?). By prioritizing raw algorithmic logic over expensive external databases like LinkedIn, we built a scalable solution that cost nearly zero to run.<\/p>\n\n\n\n
              It reinforced a valuable lesson: sometimes you don’t need a Neural Network. You just need a dictionary, a regex pattern, and a well-tuned while<\/code> loop.<\/p>\n","protected":false},"excerpt":{"rendered":"
              In the world of Private Equity, Vintage is everything. An investor looks at a company with $1M in revenue and asks: “Did they reach this in 12 months, or 12…<\/p>\n","protected":false},"author":1,"featured_media":157,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[8,4],"tags":[],"class_list":["post-77","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-learn-with-me","category-tech-posts"],"jetpack_featured_media_url":"https:\/\/balamurali.in\/blog\/wp-content\/uploads\/2026\/02\/founding_year_featured.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/posts\/77","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/comments?post=77"}],"version-history":[{"count":1,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/posts\/77\/revisions"}],"predecessor-version":[{"id":79,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/posts\/77\/revisions\/79"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/media\/157"}],"wp:attachment":[{"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/media?parent=77"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/categories?post=77"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/balamurali.in\/blog\/wp-json\/wp\/v2\/tags?post=77"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}