Audit Title and Meta Description Lengths from a Sitemap with Python (Standard Library Only)

A sitemap can be valid XML and still ship titles like Home or meta descriptions that are empty, identical across fifty product pages, or long enough to get cut mid-sentence in a SERP. None of that shows up as a sitemap error. You only see it when you open each page, or when Search Console shows a title Google rewrote for you. This post builds a small auditor in Python using only the standard library (html.parser, urllib, xml.etree, csv, concurrent.futures). It reads a sitemap or sitemap index, fetches every listed URL, extracts the and meta name="description", measures their character lengths, flags missing and empty values, and marks duplicate titles or descriptions across the set. The output is a CSV you can sort in a spreadsheet. Google Search Central docs on title links and snippets / meta descriptions were opened on 7 Nov 2026. Appie free-tool descriptions are as of 7 Nov 2026. The script was tested on Python 3.13.5 against a local fixture site, shown below. Disclosure: Appie.

Audit Title and Meta Description Lengths from a Sitemap with Python (Standard Library Only)

A sitemap can be valid XML and still ship titles like Home or meta descriptions that are empty, identical across fifty product pages, or long enough to get cut mid-sentence in a SERP. None of that shows up as a sitemap error. You only see it when you open each page, or when Search Console shows a title Google rewrote for you.

This post builds a small auditor in Python using only the standard library (html.parser, urllib, xml.etree, csv, concurrent.futures). It reads a sitemap or sitemap index, fetches every listed URL, extracts the </code> and <code>meta name="description"</code>, measures their character lengths, flags missing and empty values, and marks duplicate titles or descriptions across the set. The output is a CSV you can sort in a spreadsheet. <p><em>Google Search Central docs on <a href="https://developers.google.com/search/docs/appearance/title-link" rel="noopener noreferrer">title links</a> and <a href="https://developers.google.com/search/docs/appearance/snippet" rel="noopener noreferrer">snippets / meta descriptions</a> were opened on 7 Nov 2026. Appie free-tool descriptions are as of 7 Nov 2026. The script was tested on Python 3.13.5 against a local fixture site, shown below.</em> <p><em>Disclosure: Appie.run publishes this article. Two free tools mentioned near the end are ours. The script does not depend on them.</em> <h2> Where this fits </h2> <p>Earlier posts in this series covered <a href="https://dev.to/appierun_66/how-to-extract-and-check-json-ld-structured-data-with-python-standard-library-only-21g7">extracting JSON-LD structured data with the standard library</a> and <a href="https://dev.to/appierun_66/find-broken-links-and-redirect-chains-in-your-sitemap-with-python-standard-library-only-2alp">finding broken links and redirect chains in a sitemap</a>. Those check machine-readable markup and HTTP behavior. Titles and meta descriptions are the human-facing SERP text. Same pattern: parse what the server returns, print what is there, fail loudly when the page is incomplete or out of a practical length band. <h2> What Google actually says about length </h2> <p>This matters, because a lot of SEO checklists still treat "60 characters" or "160 characters" as if Google published those numbers as rules. The current docs do not. <p>On <a href="https://developers.google.com/search/docs/appearance/title-link" rel="noopener noreferrer">title links</a> (opened 7 Nov 2026), Google says every page should have a <code><title></code>, that the text should be descriptive and concise, and that you should avoid vague titles, keyword stuffing, and repeated boilerplate. It also says: there is <strong>no limit</strong> on how long a <code><title></code> element can be, but the title link is truncated in Google Search results as needed, typically to fit the device width. Google may also rewrite the title link using on-page headings, <code>og:title</code>, anchor text and other sources when it detects issues. <p>On <a href="https://developers.google.com/search/docs/appearance/snippet" rel="noopener noreferrer">snippets and meta descriptions</a> (opened 7 Nov 2026), Google says snippets are primarily created from page content, and that Google sometimes uses the meta description when it describes the page better. Again: there is <strong>no limit</strong> on how long a meta description can be, but the snippet is truncated as needed to fit the device. Unique, page-specific descriptions are preferred over site-wide boilerplate. Keyword lists are less likely to be shown as a snippet. The docs even call out a "too short" example as a bad pattern. <p>So this script does <strong>not</strong> claim a Google-official character cutoff. It uses a <strong>practical audit band</strong> as a starting point for triage: <ul> <li> <strong><code><title></code></strong>: Practical band (defaults): 30 to 60 characters; Why it is useful anyway: Catches <code>Home</code> / <code>Contact</code> stubs and essay-length titles that almost always truncate.</li> <li> <strong><code>meta description</code></strong>: Practical band (defaults): 70 to 160 characters; Why it is useful anyway: Catches empty pitches and keyword dumps; puts the important sentence first.</li> </ul> <p>Override the bands with flags when your brand suffix or language needs a different window. The point of the band is to surface outliers for a human edit, not to score pages against a Google rule that does not exist. <h2> The script </h2> <p>Save this as <code>title_meta_audit.py</code>. It should run on Python 3.8 or later; we tested it on 3.13.5.<br> <div class="highlight js-code-highlight"> <pre class="highlight python"><code><span class="c1">#!/usr/bin/env python3 </span><span class="sh">"""</span><span class="s">Audit <title> and meta description lengths for every URL in an XML sitemap. Standard library only. Usage: python3 title_meta_audit.py https://example.com/sitemap.xml --out report.csv python3 title_meta_audit.py /path/to/sitemap.xml --out report.csv python3 title_meta_audit.py sitemap.xml --limit 20 Practical length bands (NOT Google rules; Google documents no fixed character limit and truncates to fit the device). Defaults: title: 30-60 characters meta description: 70-160 characters Override with --title-min/--title-max/--desc-min/--desc-max. </span><span class="sh">"""</span> <span class="kn">import</span> <span class="n">argparse</span> <span class="kn">import</span> <span class="n">csv</span> <span class="kn">import</span> <span class="n">gzip</span> <span class="kn">import</span> <span class="n">sys</span> <span class="kn">import</span> <span class="n">threading</span> <span class="kn">import</span> <span class="n">time</span> <span class="kn">import</span> <span class="n">urllib.error</span> <span class="kn">import</span> <span class="n">urllib.request</span> <span class="kn">import</span> <span class="n">xml.etree.ElementTree</span> <span class="k">as</span> <span class="n">ET</span> <span class="kn">from</span> <span class="n">collections</span> <span class="kn">import</span> <span class="n">defaultdict</span> <span class="kn">from</span> <span class="n">concurrent.futures</span> <span class="kn">import</span> <span class="n">ThreadPoolExecutor</span><span class="p">,</span> <span class="n">as_completed</span> <span class="kn">from</span> <span class="n">html.parser</span> <span class="kn">import</span> <span class="n">HTMLParser</span> <span class="n">NS</span> <span class="o">=</span> <span class="sh">"</span><span class="s">{http://www.sitemaps.org/schemas/sitemap/0.9}</span><span class="sh">"</span> <span class="n">UA</span> <span class="o">=</span> <span class="sh">"</span><span class="s">title-meta-audit/1.0 (+https://example.com/contact)</span><span class="sh">"</span> <span class="k">class</span> <span class="nc">TitleMetaParser</span><span class="p">(</span><span class="n">HTMLParser</span><span class="p">):</span> <span class="sh">"""</span><span class="s">Collect <title> text and name=description meta content from <head>.</span><span class="sh">"""</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="n">self</span><span class="p">):</span> <span class="nf">super</span><span class="p">().</span><span class="nf">__init__</span><span class="p">()</span> <span class="n">self</span><span class="p">.</span><span class="n">title_parts</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">self</span><span class="p">.</span><span class="n">description</span> <span class="o">=</span> <span class="bp">None</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_head</span> <span class="o">=</span> <span class="bp">False</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_title</span> <span class="o">=</span> <span class="bp">False</span> <span class="n">self</span><span class="p">.</span><span class="n">_done</span> <span class="o">=</span> <span class="bp">False</span> <span class="k">def</span> <span class="nf">handle_starttag</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span> <span class="k">if</span> <span class="n">self</span><span class="p">.</span><span class="n">_done</span><span class="p">:</span> <span class="k">return</span> <span class="n">name</span> <span class="o">=</span> <span class="n">tag</span><span class="p">.</span><span class="nf">lower</span><span class="p">()</span> <span class="n">attr</span> <span class="o">=</span> <span class="p">{</span><span class="n">k</span><span class="p">.</span><span class="nf">lower</span><span class="p">():</span> <span class="p">(</span><span class="n">v</span> <span class="ow">or</span> <span class="sh">""</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">}</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">head</span><span class="sh">"</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_head</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">return</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">body</span><span class="sh">"</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">_done</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">return</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_head</span><span class="p">:</span> <span class="k">return</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_title</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">return</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">meta</span><span class="sh">"</span><span class="p">:</span> <span class="n">meta_name</span> <span class="o">=</span> <span class="p">(</span><span class="n">attr</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="sh">"</span><span class="s">name</span><span class="sh">"</span><span class="p">)</span> <span class="ow">or</span> <span class="sh">""</span><span class="p">).</span><span class="nf">strip</span><span class="p">().</span><span class="nf">lower</span><span class="p">()</span> <span class="k">if</span> <span class="n">meta_name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">description</span><span class="sh">"</span> <span class="ow">and</span> <span class="n">self</span><span class="p">.</span><span class="n">description</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">attr</span><span class="p">.</span><span class="nf">get</span><span class="p">(</span><span class="sh">"</span><span class="s">content</span><span class="sh">"</span><span class="p">,</span> <span class="sh">""</span><span class="p">)</span> <span class="k">def</span> <span class="nf">handle_endtag</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">tag</span><span class="p">):</span> <span class="n">name</span> <span class="o">=</span> <span class="n">tag</span><span class="p">.</span><span class="nf">lower</span><span class="p">()</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_title</span> <span class="o">=</span> <span class="bp">False</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="sh">"</span><span class="s">head</span><span class="sh">"</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_head</span> <span class="o">=</span> <span class="bp">False</span> <span class="n">self</span><span class="p">.</span><span class="n">_done</span> <span class="o">=</span> <span class="bp">True</span> <span class="k">def</span> <span class="nf">handle_data</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span> <span class="k">if</span> <span class="n">self</span><span class="p">.</span><span class="n">_in_title</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">self</span><span class="p">.</span><span class="n">_done</span><span class="p">:</span> <span class="n">self</span><span class="p">.</span><span class="n">title_parts</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> <span class="nd">@property</span> <span class="k">def</span> <span class="nf">title</span><span class="p">(</span><span class="n">self</span><span class="p">):</span> <span class="k">return</span> <span class="sh">""</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">self</span><span class="p">.</span><span class="n">title_parts</span><span class="p">).</span><span class="nf">strip</span><span class="p">()</span> <span class="k">def</span> <span class="nf">fetch_bytes</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="n">timeout</span><span class="p">):</span> <span class="k">if</span> <span class="n">src</span><span class="p">.</span><span class="nf">startswith</span><span class="p">((</span><span class="sh">"</span><span class="s">http://</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">https://</span><span class="sh">"</span><span class="p">)):</span> <span class="n">req</span> <span class="o">=</span> <span class="n">urllib</span><span class="p">.</span><span class="n">request</span><span class="p">.</span><span class="nc">Request</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">User-Agent</span><span class="sh">"</span><span class="p">:</span> <span class="n">UA</span><span class="p">})</span> <span class="k">with</span> <span class="n">urllib</span><span class="p">.</span><span class="n">request</span><span class="p">.</span><span class="nf">urlopen</span><span class="p">(</span><span class="n">req</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="n">timeout</span><span class="p">)</span> <span class="k">as</span> <span class="n">resp</span><span class="p">:</span> <span class="n">data</span> <span class="o">=</span> <span class="n">resp</span><span class="p">.</span><span class="nf">read</span><span class="p">()</span> <span class="k">else</span><span class="p">:</span> <span class="k">with</span> <span class="nf">open</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="sh">"</span><span class="s">rb</span><span class="sh">"</span><span class="p">)</span> <span class="k">as</span> <span class="n">fh</span><span class="p">:</span> <span class="n">data</span> <span class="o">=</span> <span class="n">fh</span><span class="p">.</span><span class="nf">read</span><span class="p">()</span> <span class="k">return</span> <span class="n">gzip</span><span class="p">.</span><span class="nf">decompress</span><span class="p">(</span><span class="n">data</span><span class="p">)</span> <span class="k">if</span> <span class="n">data</span><span class="p">[:</span><span class="mi">2</span><span class="p">]</span> <span class="o">==</span> <span class="sa">b</span><span class="sh">"</span><span class="se">\x1f\x8b</span><span class="sh">"</span> <span class="k">else</span> <span class="n">data</span> <span class="k">def</span> <span class="nf">sitemap_urls</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="n">timeout</span><span class="p">,</span> <span class="n">seen</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">depth</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span> <span class="n">seen</span> <span class="o">=</span> <span class="n">seen</span> <span class="k">if</span> <span class="n">seen</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="nf">set</span><span class="p">()</span> <span class="k">if</span> <span class="n">src</span> <span class="ow">in</span> <span class="n">seen</span> <span class="ow">or</span> <span class="n">depth</span> <span class="o">></span> <span class="mi">3</span><span class="p">:</span> <span class="k">return</span> <span class="p">[]</span> <span class="n">seen</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="n">src</span><span class="p">)</span> <span class="n">root</span> <span class="o">=</span> <span class="n">ET</span><span class="p">.</span><span class="nf">fromstring</span><span class="p">(</span><span class="nf">fetch_bytes</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="n">timeout</span><span class="p">))</span> <span class="n">locs</span> <span class="o">=</span> <span class="p">[</span><span class="n">el</span><span class="p">.</span><span class="n">text</span><span class="p">.</span><span class="nf">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">el</span> <span class="ow">in</span> <span class="n">root</span><span class="p">.</span><span class="nf">iter</span><span class="p">(</span><span class="n">NS</span> <span class="o">+</span> <span class="sh">"</span><span class="s">loc</span><span class="sh">"</span><span class="p">)</span> <span class="k">if</span> <span class="n">el</span><span class="p">.</span><span class="n">text</span><span class="p">]</span> <span class="k">if</span> <span class="n">root</span><span class="p">.</span><span class="n">tag</span> <span class="o">==</span> <span class="n">NS</span> <span class="o">+</span> <span class="sh">"</span><span class="s">sitemapindex</span><span class="sh">"</span><span class="p">:</span> <span class="n">urls</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">for</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">locs</span><span class="p">:</span> <span class="n">urls</span> <span class="o">+=</span> <span class="nf">sitemap_urls</span><span class="p">(</span><span class="n">child</span><span class="p">,</span> <span class="n">timeout</span><span class="p">,</span> <span class="n">seen</span><span class="p">,</span> <span class="n">depth</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span> <span class="k">return</span> <span class="n">urls</span> <span class="k">return</span> <span class="n">locs</span> <span class="k">def</span> <span class="nf">fetch_html</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="p">):</span> <span class="n">req</span> <span class="o">=</span> <span class="n">urllib</span><span class="p">.</span><span class="n">request</span><span class="p">.</span><span class="nc">Request</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="sh">"</span><span class="s">User-Agent</span><span class="sh">"</span><span class="p">:</span> <span class="n">UA</span><span class="p">})</span> <span class="k">with</span> <span class="n">urllib</span><span class="p">.</span><span class="n">request</span><span class="p">.</span><span class="nf">urlopen</span><span class="p">(</span><span class="n">req</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="n">timeout</span><span class="p">)</span> <span class="k">as</span> <span class="n">resp</span><span class="p">:</span> <span class="n">charset</span> <span class="o">=</span> <span class="n">resp</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="nf">get_content_charset</span><span class="p">()</span> <span class="ow">or</span> <span class="sh">"</span><span class="s">utf-8</span><span class="sh">"</span> <span class="k">return</span> <span class="n">resp</span><span class="p">.</span><span class="nf">read</span><span class="p">().</span><span class="nf">decode</span><span class="p">(</span><span class="n">charset</span><span class="p">,</span> <span class="n">errors</span><span class="o">=</span><span class="sh">"</span><span class="s">replace</span><span class="sh">"</span><span class="p">),</span> <span class="n">resp</span><span class="p">.</span><span class="nf">geturl</span><span class="p">(),</span> <span class="n">resp</span><span class="p">.</span><span class="n">status</span> <span class="k">def</span> <span class="nf">classify</span><span class="p">(</span><span class="n">title</span><span class="p">,</span> <span class="n">description</span><span class="p">,</span> <span class="n">title_min</span><span class="p">,</span> <span class="n">title_max</span><span class="p">,</span> <span class="n">desc_min</span><span class="p">,</span> <span class="n">desc_max</span><span class="p">):</span> <span class="n">issues</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">title_len</span> <span class="o">=</span> <span class="nf">len</span><span class="p">(</span><span class="n">title</span><span class="p">)</span> <span class="k">if</span> <span class="n">title</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="mi">0</span> <span class="n">desc_len</span> <span class="o">=</span> <span class="nf">len</span><span class="p">(</span><span class="n">description</span><span class="p">)</span> <span class="k">if</span> <span class="n">description</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="mi">0</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">title</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sh">"</span><span class="s">missing title</span><span class="sh">"</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="k">if</span> <span class="n">title_len</span> <span class="o"><</span> <span class="n">title_min</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sa">f</span><span class="sh">"</span><span class="s">title too short (</span><span class="si">{</span><span class="n">title_len</span><span class="si">}</span><span class="s"> < </span><span class="si">{</span><span class="n">title_min</span><span class="si">}</span><span class="s">)</span><span class="sh">"</span><span class="p">)</span> <span class="k">elif</span> <span class="n">title_len</span> <span class="o">></span> <span class="n">title_max</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sa">f</span><span class="sh">"</span><span class="s">title too long (</span><span class="si">{</span><span class="n">title_len</span><span class="si">}</span><span class="s"> > </span><span class="si">{</span><span class="n">title_max</span><span class="si">}</span><span class="s">)</span><span class="sh">"</span><span class="p">)</span> <span class="k">if</span> <span class="n">description</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sh">"</span><span class="s">missing meta description</span><span class="sh">"</span><span class="p">)</span> <span class="k">elif</span> <span class="n">description</span><span class="p">.</span><span class="nf">strip</span><span class="p">()</span> <span class="o">==</span> <span class="sh">""</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sh">"</span><span class="s">empty meta description</span><span class="sh">"</span><span class="p">)</span> <span class="k">else</span><span class="p">:</span> <span class="k">if</span> <span class="n">desc_len</span> <span class="o"><</span> <span class="n">desc_min</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sa">f</span><span class="sh">"</span><span class="s">description too short (</span><span class="si">{</span><span class="n">desc_len</span><span class="si">}</span><span class="s"> < </span><span class="si">{</span><span class="n">desc_min</span><span class="si">}</span><span class="s">)</span><span class="sh">"</span><span class="p">)</span> <span class="k">elif</span> <span class="n">desc_len</span> <span class="o">></span> <span class="n">desc_max</span><span class="p">:</span> <span class="n">issues</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sa">f</span><span class="sh">"</span><span class="s">description too long (</span><span class="si">{</span><span class="n">desc_len</span><span class="si">}</span><span class="s"> > </span><span class="si">{</span><span class="n">desc_max</span><span class="si">}</span><span class="s">)</span><span class="sh">"</span><span class="p">)</span> <span class="k">return</span> <span class="n">issues</span><span class="p">,</span> <span class="n">title_len</span><span class="p">,</span> <span class="n">desc_len</span> <span class="k">def</span> <span class="nf">audit_one</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="p">,</span> <span class="n">bands</span><span class="p">):</span> <span class="n">title_min</span><span class="p">,</span> <span class="n">title_max</span><span class="p">,</span> <span class="n">desc_min</span><span class="p">,</span> <span class="n">desc_max</span> <span class="o">=</span> <span class="n">bands</span> <span class="k">try</span><span class="p">:</span> <span class="n">html</span><span class="p">,</span> <span class="n">final_url</span><span class="p">,</span> <span class="n">status</span> <span class="o">=</span> <span class="nf">fetch_html</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">timeout</span><span class="p">)</span> <span class="k">except</span> <span class="n">urllib</span><span class="p">.</span><span class="n">error</span><span class="p">.</span><span class="n">HTTPError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="k">return</span> <span class="p">{</span> <span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">:</span> <span class="n">url</span><span class="p">,</span> <span class="sh">"</span><span class="s">final_url</span><span class="sh">"</span><span class="p">:</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">status</span><span class="sh">"</span><span class="p">:</span> <span class="n">e</span><span class="p">.</span><span class="n">code</span><span class="p">,</span> <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">title_len</span><span class="sh">"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">:</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">desc_len</span><span class="sh">"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">:</span> <span class="sa">f</span><span class="sh">"</span><span class="s">HTTP error </span><span class="si">{</span><span class="n">e</span><span class="p">.</span><span class="n">code</span><span class="si">}</span><span class="sh">"</span><span class="p">,</span> <span class="p">}</span> <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="k">return</span> <span class="p">{</span> <span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">:</span> <span class="n">url</span><span class="p">,</span> <span class="sh">"</span><span class="s">final_url</span><span class="sh">"</span><span class="p">:</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">status</span><span class="sh">"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">title_len</span><span class="sh">"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">:</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">desc_len</span><span class="sh">"</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">:</span> <span class="sa">f</span><span class="sh">"</span><span class="s">error: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="sh">"</span><span class="p">,</span> <span class="p">}</span> <span class="n">parser</span> <span class="o">=</span> <span class="nc">TitleMetaParser</span><span class="p">()</span> <span class="n">parser</span><span class="p">.</span><span class="nf">feed</span><span class="p">(</span><span class="n">html</span><span class="p">)</span> <span class="n">title</span> <span class="o">=</span> <span class="n">parser</span><span class="p">.</span><span class="n">title</span> <span class="n">description</span> <span class="o">=</span> <span class="n">parser</span><span class="p">.</span><span class="n">description</span> <span class="n">issues</span><span class="p">,</span> <span class="n">title_len</span><span class="p">,</span> <span class="n">desc_len</span> <span class="o">=</span> <span class="nf">classify</span><span class="p">(</span> <span class="n">title</span><span class="p">,</span> <span class="n">description</span><span class="p">,</span> <span class="n">title_min</span><span class="p">,</span> <span class="n">title_max</span><span class="p">,</span> <span class="n">desc_min</span><span class="p">,</span> <span class="n">desc_max</span> <span class="p">)</span> <span class="k">return</span> <span class="p">{</span> <span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">:</span> <span class="n">url</span><span class="p">,</span> <span class="sh">"</span><span class="s">final_url</span><span class="sh">"</span><span class="p">:</span> <span class="n">final_url</span><span class="p">,</span> <span class="sh">"</span><span class="s">status</span><span class="sh">"</span><span class="p">:</span> <span class="n">status</span><span class="p">,</span> <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span> <span class="sh">"</span><span class="s">title_len</span><span class="sh">"</span><span class="p">:</span> <span class="n">title_len</span><span class="p">,</span> <span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">:</span> <span class="n">description</span> <span class="k">if</span> <span class="n">description</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="sh">""</span><span class="p">,</span> <span class="sh">"</span><span class="s">desc_len</span><span class="sh">"</span><span class="p">:</span> <span class="n">desc_len</span><span class="p">,</span> <span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">:</span> <span class="sh">"</span><span class="s">; </span><span class="sh">"</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">issues</span><span class="p">)</span> <span class="k">if</span> <span class="n">issues</span> <span class="k">else</span> <span class="sh">""</span><span class="p">,</span> <span class="p">}</span> <span class="k">def</span> <span class="nf">main</span><span class="p">(</span><span class="n">argv</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span> <span class="n">p</span> <span class="o">=</span> <span class="n">argparse</span><span class="p">.</span><span class="nc">ArgumentParser</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="n">__doc__</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">sitemap</span><span class="sh">"</span><span class="p">,</span> <span class="nb">help</span><span class="o">=</span><span class="sh">"</span><span class="s">Sitemap URL or local XML path</span><span class="sh">"</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--out</span><span class="sh">"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="sh">"</span><span class="s">title_meta_report.csv</span><span class="sh">"</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--timeout</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">float</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mf">15.0</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--workers</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">int</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--delay</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">float</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mf">0.25</span><span class="p">,</span> <span class="nb">help</span><span class="o">=</span><span class="sh">"</span><span class="s">Seconds between request starts</span><span class="sh">"</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--limit</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">int</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="nb">help</span><span class="o">=</span><span class="sh">"</span><span class="s">Audit at most N URLs (0 = all)</span><span class="sh">"</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--title-min</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">int</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mi">30</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--title-max</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">int</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mi">60</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--desc-min</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">int</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mi">70</span><span class="p">)</span> <span class="n">p</span><span class="p">.</span><span class="nf">add_argument</span><span class="p">(</span><span class="sh">"</span><span class="s">--desc-max</span><span class="sh">"</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="nb">int</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="mi">160</span><span class="p">)</span> <span class="n">args</span> <span class="o">=</span> <span class="n">p</span><span class="p">.</span><span class="nf">parse_args</span><span class="p">(</span><span class="n">argv</span><span class="p">)</span> <span class="n">bands</span> <span class="o">=</span> <span class="p">(</span><span class="n">args</span><span class="p">.</span><span class="n">title_min</span><span class="p">,</span> <span class="n">args</span><span class="p">.</span><span class="n">title_max</span><span class="p">,</span> <span class="n">args</span><span class="p">.</span><span class="n">desc_min</span><span class="p">,</span> <span class="n">args</span><span class="p">.</span><span class="n">desc_max</span><span class="p">)</span> <span class="k">try</span><span class="p">:</span> <span class="n">urls</span> <span class="o">=</span> <span class="nf">sitemap_urls</span><span class="p">(</span><span class="n">args</span><span class="p">.</span><span class="n">sitemap</span><span class="p">,</span> <span class="n">args</span><span class="p">.</span><span class="n">timeout</span><span class="p">)</span> <span class="k">except</span> <span class="nb">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> <span class="nf">print</span><span class="p">(</span><span class="sa">f</span><span class="sh">"</span><span class="s">Failed to read sitemap: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="sh">"</span><span class="p">,</span> <span class="nb">file</span><span class="o">=</span><span class="n">sys</span><span class="p">.</span><span class="n">stderr</span><span class="p">)</span> <span class="k">return</span> <span class="mi">2</span> <span class="k">if</span> <span class="n">args</span><span class="p">.</span><span class="n">limit</span> <span class="ow">and</span> <span class="n">args</span><span class="p">.</span><span class="n">limit</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span> <span class="n">urls</span> <span class="o">=</span> <span class="n">urls</span><span class="p">[:</span> <span class="n">args</span><span class="p">.</span><span class="n">limit</span><span class="p">]</span> <span class="nf">print</span><span class="p">(</span> <span class="sa">f</span><span class="sh">"</span><span class="s">auditing </span><span class="si">{</span><span class="nf">len</span><span class="p">(</span><span class="n">urls</span><span class="p">)</span><span class="si">}</span><span class="s"> URLs with </span><span class="si">{</span><span class="n">args</span><span class="p">.</span><span class="n">workers</span><span class="si">}</span><span class="s"> workers </span><span class="sh">"</span> <span class="sa">f</span><span class="sh">"</span><span class="s">(title </span><span class="si">{</span><span class="n">args</span><span class="p">.</span><span class="n">title_min</span><span class="si">}</span><span class="s">-</span><span class="si">{</span><span class="n">args</span><span class="p">.</span><span class="n">title_max</span><span class="si">}</span><span class="s">, </span><span class="sh">"</span> <span class="sa">f</span><span class="sh">"</span><span class="s">description </span><span class="si">{</span><span class="n">args</span><span class="p">.</span><span class="n">desc_min</span><span class="si">}</span><span class="s">-</span><span class="si">{</span><span class="n">args</span><span class="p">.</span><span class="n">desc_max</span><span class="si">}</span><span class="s">; practical bands)</span><span class="sh">"</span> <span class="p">)</span> <span class="n">lock</span> <span class="o">=</span> <span class="n">threading</span><span class="p">.</span><span class="nc">Lock</span><span class="p">()</span> <span class="n">last_start</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.0</span><span class="p">]</span> <span class="n">rows</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">def</span> <span class="nf">gated</span><span class="p">(</span><span class="n">url</span><span class="p">):</span> <span class="k">with</span> <span class="n">lock</span><span class="p">:</span> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="p">.</span><span class="nf">monotonic</span><span class="p">()</span> <span class="n">wait</span> <span class="o">=</span> <span class="n">args</span><span class="p">.</span><span class="n">delay</span> <span class="o">-</span> <span class="p">(</span><span class="n">now</span> <span class="o">-</span> <span class="n">last_start</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">wait</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span> <span class="n">time</span><span class="p">.</span><span class="nf">sleep</span><span class="p">(</span><span class="n">wait</span><span class="p">)</span> <span class="n">last_start</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">time</span><span class="p">.</span><span class="nf">monotonic</span><span class="p">()</span> <span class="k">return</span> <span class="nf">audit_one</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">args</span><span class="p">.</span><span class="n">timeout</span><span class="p">,</span> <span class="n">bands</span><span class="p">)</span> <span class="k">with</span> <span class="nc">ThreadPoolExecutor</span><span class="p">(</span><span class="n">max_workers</span><span class="o">=</span><span class="n">args</span><span class="p">.</span><span class="n">workers</span><span class="p">)</span> <span class="k">as</span> <span class="n">pool</span><span class="p">:</span> <span class="n">futures</span> <span class="o">=</span> <span class="p">{</span><span class="n">pool</span><span class="p">.</span><span class="nf">submit</span><span class="p">(</span><span class="n">gated</span><span class="p">,</span> <span class="n">u</span><span class="p">):</span> <span class="n">u</span> <span class="k">for</span> <span class="n">u</span> <span class="ow">in</span> <span class="n">urls</span><span class="p">}</span> <span class="k">for</span> <span class="n">fut</span> <span class="ow">in</span> <span class="nf">as_completed</span><span class="p">(</span><span class="n">futures</span><span class="p">):</span> <span class="n">rows</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="n">fut</span><span class="p">.</span><span class="nf">result</span><span class="p">())</span> <span class="c1"># Preserve sitemap order </span> <span class="n">by_url</span> <span class="o">=</span> <span class="p">{</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">]:</span> <span class="n">r</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">rows</span><span class="p">}</span> <span class="n">ordered</span> <span class="o">=</span> <span class="p">[</span><span class="n">by_url</span><span class="p">[</span><span class="n">u</span><span class="p">]</span> <span class="k">for</span> <span class="n">u</span> <span class="ow">in</span> <span class="n">urls</span> <span class="k">if</span> <span class="n">u</span> <span class="ow">in</span> <span class="n">by_url</span><span class="p">]</span> <span class="c1"># Duplicate title / description across pages (after all fetches) </span> <span class="n">title_map</span> <span class="o">=</span> <span class="nf">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span> <span class="n">desc_map</span> <span class="o">=</span> <span class="nf">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">ordered</span><span class="p">:</span> <span class="k">if</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">]:</span> <span class="n">title_map</span><span class="p">[</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">]].</span><span class="nf">append</span><span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">])</span> <span class="k">if</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">]:</span> <span class="n">desc_map</span><span class="p">[</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">]].</span><span class="nf">append</span><span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">])</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">ordered</span><span class="p">:</span> <span class="n">extras</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">if</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">]</span> <span class="ow">and</span> <span class="nf">len</span><span class="p">(</span><span class="n">title_map</span><span class="p">[</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">]])</span> <span class="o">></span> <span class="mi">1</span><span class="p">:</span> <span class="n">extras</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sh">"</span><span class="s">duplicate title</span><span class="sh">"</span><span class="p">)</span> <span class="k">if</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">]</span> <span class="ow">and</span> <span class="nf">len</span><span class="p">(</span><span class="n">desc_map</span><span class="p">[</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">]])</span> <span class="o">></span> <span class="mi">1</span><span class="p">:</span> <span class="n">extras</span><span class="p">.</span><span class="nf">append</span><span class="p">(</span><span class="sh">"</span><span class="s">duplicate description</span><span class="sh">"</span><span class="p">)</span> <span class="k">if</span> <span class="n">extras</span><span class="p">:</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span> <span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">]</span> <span class="o">+</span> <span class="sh">"</span><span class="s">; </span><span class="sh">"</span> <span class="k">if</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">]</span> <span class="k">else</span> <span class="sh">""</span><span class="p">)</span> <span class="o">+</span> <span class="sh">"</span><span class="s">; </span><span class="sh">"</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">extras</span><span class="p">)</span> <span class="p">)</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">ordered</span><span class="p">:</span> <span class="n">flag</span> <span class="o">=</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">]</span> <span class="ow">or</span> <span class="sh">"</span><span class="s">ok</span><span class="sh">"</span> <span class="nf">print</span><span class="p">(</span> <span class="sa">f</span><span class="sh">"</span><span class="si">{</span><span class="n">r</span><span class="p">[</span><span class="sh">'</span><span class="s">status</span><span class="sh">'</span><span class="p">]</span><span class="si">}</span><span class="s"> </span><span class="si">{</span><span class="n">r</span><span class="p">[</span><span class="sh">'</span><span class="s">url</span><span class="sh">'</span><span class="p">]</span><span class="si">}</span><span class="s"> </span><span class="sh">"</span> <span class="sa">f</span><span class="sh">"</span><span class="s">title=</span><span class="si">{</span><span class="n">r</span><span class="p">[</span><span class="sh">'</span><span class="s">title_len</span><span class="sh">'</span><span class="p">]</span><span class="si">}</span><span class="s"> desc=</span><span class="si">{</span><span class="n">r</span><span class="p">[</span><span class="sh">'</span><span class="s">desc_len</span><span class="sh">'</span><span class="p">]</span><span class="si">}</span><span class="s"> </span><span class="si">{</span><span class="n">flag</span><span class="si">}</span><span class="sh">"</span> <span class="p">)</span> <span class="n">fields</span> <span class="o">=</span> <span class="p">[</span> <span class="sh">"</span><span class="s">url</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">final_url</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">status</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">title</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">title_len</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">description</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">desc_len</span><span class="sh">"</span><span class="p">,</span> <span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">,</span> <span class="p">]</span> <span class="k">with</span> <span class="nf">open</span><span class="p">(</span><span class="n">args</span><span class="p">.</span><span class="n">out</span><span class="p">,</span> <span class="sh">"</span><span class="s">w</span><span class="sh">"</span><span class="p">,</span> <span class="n">newline</span><span class="o">=</span><span class="sh">""</span><span class="p">,</span> <span class="n">encoding</span><span class="o">=</span><span class="sh">"</span><span class="s">utf-8</span><span class="sh">"</span><span class="p">)</span> <span class="k">as</span> <span class="n">fh</span><span class="p">:</span> <span class="n">w</span> <span class="o">=</span> <span class="n">csv</span><span class="p">.</span><span class="nc">DictWriter</span><span class="p">(</span><span class="n">fh</span><span class="p">,</span> <span class="n">fieldnames</span><span class="o">=</span><span class="n">fields</span><span class="p">)</span> <span class="n">w</span><span class="p">.</span><span class="nf">writeheader</span><span class="p">()</span> <span class="n">w</span><span class="p">.</span><span class="nf">writerows</span><span class="p">(</span><span class="n">ordered</span><span class="p">)</span> <span class="n">ok</span> <span class="o">=</span> <span class="nf">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">r</span> <span class="ow">in</span> <span class="n">ordered</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">r</span><span class="p">[</span><span class="sh">"</span><span class="s">issues</span><span class="sh">"</span><span class="p">])</span> <span class="n">bad</span> <span class="o">=</span> <span class="nf">len</span><span class="p">(</span><span class="n">ordered</span><span class="p">)</span> <span class="o">-</span> <span class="n">ok</span> <span class="nf">print</span><span class="p">(</span><span class="sa">f</span><span class="sh">"</span><span class="si">{</span><span class="n">ok</span><span class="si">}</span><span class="s"> ok, </span><span class="si">{</span><span class="n">bad</span><span class="si">}</span><span class="s"> with issues; report: </span><span class="si">{</span><span class="n">args</span><span class="p">.</span><span class="n">out</span><span class="si">}</span><span class="sh">"</span><span class="p">)</span> <span class="nf">print</span><span class="p">(</span> <span class="sh">"</span><span class="s">Note: length bands are a practical audit starting point, not a Google rule. </span><span class="sh">"</span> <span class="sh">"</span><span class="s">Google documents no fixed character limit for titles or meta descriptions.</span><span class="sh">"</span> <span class="p">)</span> <span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="n">bad</span> <span class="k">else</span> <span class="mi">0</span> <span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="sh">"</span><span class="s">__main__</span><span class="sh">"</span><span class="p">:</span> <span class="n">sys</span><span class="p">.</span><span class="nf">exit</span><span class="p">(</span><span class="nf">main</span><span class="p">())</span> </code></pre> </div> <h2> Local test </h2> <p>Serve a small fixture site with pages that miss titles, miss descriptions, go too short or too long, and share duplicate SERP text. Example pages (abbreviated):<br> <div class="highlight js-code-highlight"> <pre class="highlight html"><code><span class="c"><!-- good.html: title 43, description 125 --></span> <span class="nt"><title></span>Pricing Plans for Growing SaaS Teams | Acme<span class="nt"> name="description" content="Compare Free, Starter and Standard plans. See credit limits, included tools and when to upgrade as your support volume grows."> </span>Home<span class="nt"> name="description" content="Welcome to Acme, the product platform that helps product and GTM teams ship clearer release notes and track customer feedback in one place."> </span>Acme Blog: Complete Guide to Customer Discovery Interviews, SAFE Scripts, Note Templates and Follow-up Emails for Founders in 2026<span class="nt"> </span>About Acme: Our Story and Team<span class="nt"> </span>Contact Acme Support and Sales<span class="nt"> name="description" content="Get in touch."> </span>Acme Features Overview for Product Teams<span class="nt"> name="description" content="Explore every Acme feature in depth: ...without reinventing the wheel every sprint."> name="description" content="Docs for integrating Acme with your stack, including API keys, webhooks and MCP server setup for agent workflows."> </span>Careers at Acme: Open Roles<span class="nt"> name="description" content=""> </span>Acme Product Updates and Changelog<span class="nt"> name="description" content="Release notes and product updates from the Acme team. See what shipped this month and what is coming next for customers.">

A sitemap.xml lists those ten pages plus a deliberate 404 (missing.html). A tiny http.server on 127.0.0.1:8766 serves the folder.

python3 test_server.py
python3 title_meta_audit.py http://127.0.0.1:8766/sitemap.xml --out report.csv --delay 0.05

Real output from Python 3.13.5 on 7 Nov 2026 (exit code 1 because issues were found):

auditing 11 URLs with 4 workers (title 30-60, description 70-160; practical bands)
200 http://127.0.0.1:8766/good.html  title=43 desc=125  ok
200 http://127.0.0.1:8766/short-title.html  title=4 desc=139  title too short (4 < 30)
200 http://127.0.0.1:8766/long-title.html  title=130 desc=135  title too long (130 > 60)
200 http://127.0.0.1:8766/no-desc.html  title=30 desc=0  missing meta description
200 http://127.0.0.1:8766/short-desc.html  title=30 desc=13  description too short (13 < 70)
200 http://127.0.0.1:8766/long-desc.html  title=40 desc=423  description too long (423 > 160)
200 http://127.0.0.1:8766/no-title.html  title=0 desc=113  missing title
200 http://127.0.0.1:8766/empty-desc.html  title=27 desc=0  title too short (27 < 30); empty meta description
200 http://127.0.0.1:8766/dup-a.html  title=34 desc=120  duplicate title; duplicate description
200 http://127.0.0.1:8766/dup-b.html  title=34 desc=120  duplicate title; duplicate description
404 http://127.0.0.1:8766/missing.html  title=0 desc=0  HTTP error 404
1 ok, 10 with issues; report: /workspace/tmp/title-meta-audit/report.csv
Note: length bands are a practical audit starting point, not a Google rule. Google documents no fixed character limit for titles or meta descriptions.

Only good.html passed the default band with unique text. The two changelog pages passed the length checks on their own, then failed the duplicate pass once both had been fetched. That second pass is why the script waits until every URL is in before printing the final flags.

CSV columns: url, final_url, status, title, title_len, description, desc_len, issues. Sample rows:

url,status,title_len,desc_len,issues
http://127.0.0.1:8766/good.html,200,43,125,
http://127.0.0.1:8766/short-title.html,200,4,139,title too short (4 < 30)
http://127.0.0.1:8766/long-desc.html,200,40,423,description too long (423 > 160)
http://127.0.0.1:8766/dup-a.html,200,34,120,duplicate title; duplicate description
http://127.0.0.1:8766/missing.html,404,0,0,HTTP error 404

How the parser works

html.parser.HTMLParser only collects while it is inside , and it stops at or . That keeps footer widgets and body copy from polluting the report.

The title is the concatenated text nodes inside </code>, stripped. The description is the first <code>meta</code> whose <code>name</code> is <code>description</code> (case-insensitive). Later duplicate description tags are ignored; fix the first one. <p>Character length is Python's <code>len()</code> on the Unicode string after decode. That is a character count, not a rendered pixel width. Google truncates by device width, not by a published character quota, which is another reason the bands are labeled as a practical starting point. <p>Sitemap reading matches the earlier posts in this series: <code>xml.etree</code> walks <code>{http://www.sitemaps.org/schemas/sitemap/0.9}loc</code> nodes, follows a sitemap index up to depth 3, and accepts <code>.gz</code> payloads. Local files and <code>http(s)</code> URLs both work as the sitemap argument. <h2> Reading the report </h2> <p>A few rules of thumb once you have the CSV: <ul> <li> <strong>Missing or empty title.</strong> Google will invent a title link from headings, <code>og:title</code> or anchors. Write a real <code><title></code> that matches the main visible heading.</li> <li> <strong>Title too short.</strong> <code>Home</code>, <code>Blog</code>, <code>Pricing</code> and similar stubs fail the lower band on purpose. Add the page topic and a short brand suffix.</li> <li> <strong>Title too long.</strong> Put the distinctive part first. Brand suffixes are fine; keyword essays are not. Google may rewrite verbose titles anyway.</li> <li> <strong>Missing or empty meta description.</strong> Not a ranking penalty by itself, but you give up a chance to pitch the page. Prefer a unique sentence over nothing.</li> <li> <strong>Description too short or too long.</strong> Aim for a readable pitch with the important claim first. Truncation is width-based; front-loading still helps.</li> <li> <strong>Duplicate title or description.</strong> Common after CMS templates reuse one string. Google's title and snippet docs both warn against boilerplate. Differentiate the pages people cannot tell apart in results.</li> <li> <strong>HTTP errors.</strong> Fix or remove those URLs from the sitemap before you worry about meta length. Pair this script with the <a href="https://dev.to/appierun_66/find-broken-links-and-redirect-chains-in-your-sitemap-with-python-standard-library-only-2alp">broken-link / redirect-chain checker</a> if status codes are noisy.</li> </ul> <h2> Polite defaults and running it on a real site </h2> <p>Defaults: 4 workers, 0.25 seconds between request starts, 15 second timeout. Raise <code>--workers</code> and lower <code>--delay</code> on your own staging box if the server can take it. On a client site, ask first and keep the defaults. Change the contact URL in the <code>UA</code> string so the site owner can reach you.<br> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>python3 title_meta_audit.py https://www.example.com/sitemap.xml <span class="nt">--limit</span> 50 <span class="nt">--out</span> sample.csv python3 title_meta_audit.py https://www.example.com/sitemap.xml <span class="nt">--title-min</span> 25 <span class="nt">--title-max</span> 70 <span class="nt">--desc-min</span> 50 <span class="nt">--desc-max</span> 180 <span class="nt">--out</span> custom-bands.csv </code></pre> </div> <p>Use <code>--limit</code> for a first pass. For large indexes, audit one child sitemap at a time. <h2> Known limits </h2> <ul> <li> <strong>Server HTML only.</strong> Titles or descriptions injected only by client-side JavaScript will not appear. View-source what the server returns.</li> <li> <strong>Character count, not pixels.</strong> Wide characters and different fonts change how much fits in a SERP. The band is a triage tool.</li> <li> <strong>First <code>meta name="description"</code> only.</strong> Unusual pages with several description tags need a human look.</li> <li> <strong>No Open Graph / Twitter Card check.</strong> Those are separate preview signals; use the OG/Twitter checker from this series for link previews.</li> <li> <strong>No robots.txt handling.</strong> The script requests what the sitemap lists.</li> <li> <strong>Duplicate detection is exact.</strong> Trailing spaces or tiny copy edits count as different strings.</li> </ul> <h2> Free tools for the steps around the script </h2> <p>If you want a URL list in the browser before you run anything: <ul> <li> <strong><a href="https://appie.run/tools/sitemap-url-extractor" rel="noopener noreferrer">Sitemap URL Extractor</a></strong> (ours, free; checked 7 Nov 2026). Paste sitemap XML or enter a sitemap URL. You get one URL per line from <code>urlset</code> <code><loc></code> tags. Given a sitemap index, it lists child sitemap locations; open a child and run it again for that file's page URLs. Useful when you want to sample a subset into a shell loop before a full audit.</li> </ul> <p>When a page's meta description is empty or too thin and you need draft Q&A from the page body itself: <ul> <li> <strong><a href="https://appie.run/tools/webpage-to-faq-generator" rel="noopener noreferrer">Webpage to FAQ Generator</a></strong> (ours, free; checked 7 Nov 2026). Paste a single public page URL. It builds FAQs from the text on that page and is instructed to skip claims the page does not make. That will not write your meta description for you, but it is a fast way to surface page-specific sentences you can turn into a unique description.</li> </ul> <p>Both ask for a captcha before each run. Neither replaces the length audit this script performs. <h2> FAQ </h2> <p><strong>Is a title over 60 characters "wrong"?</strong><br> Not by Google's docs. Google documents no fixed character limit and truncates to fit the device. The default upper band is a practical flag so you review titles that will almost always truncate. Raise <code>--title-max</code> if your brand needs more room. <p><strong>Does a missing meta description hurt rankings?</strong><br> Google does not require one. It may still generate a snippet from page content. A unique description is still worth writing for pages you care about in search results. <p><strong>Why flag duplicates if both pages are "in band"?</strong><br> Because identical titles and descriptions make results hard to tell apart. Google's own title and snippet guidance warns against repeated or boilerplate text across pages. <p><strong>Can I use Beautiful Soup or lxml?</strong><br> Yes, if you already depend on them. This series stays on the standard library so the script runs anywhere Python runs, with no <code>pip install</code>. <p><strong>Will it see meta tags added by React or Vue after load?</strong><br> Only if they are present in the HTML response. Most crawlers see the first response body. <p><strong>Can I run it in CI?</strong><br> Yes. It exits with <code>1</code> when any URL has an issue, <code>2</code> when the sitemap cannot be read, and <code>0</code> when every URL is clean under your chosen bands. <h2> Wrap-up </h2> <p>Title and meta description problems rarely show up in sitemap validators. They show up as vague SERP text, Google-rewritten titles, and pages that look the same in results. Run this auditor after template changes, migrations and bulk CMS edits. Pair it with the JSON-LD and broken-link scripts when you want a fuller technical pass. If you need a URL list from a sitemap in the browser first, the free <a href="https://appie.run/tools/sitemap-url-extractor" rel="noopener noreferrer">Sitemap URL Extractor</a> is enough to get started. <p>RG, Founder, Appie.run <div class="d-flex justify-content-end mt-3"> <a href="https://dev.to/appierun_66/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only-eni" class="btn btn-default text-decoration-none" target="_blank" rel="nofollow"> Read More   <i class="icon-long-arrow-right"></i> </a> </div> </div> <div class="post-tags mt-3 mb-5"> </div> <div class="post-share mb-4"> <h3 class="sub-section-title">Share</h3> <div class="d-flex align-items-center flex-wrap gap-2"> <button type="button" onclick="window.open('https://www.facebook.com/sharer/sharer.php?u=https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only', 'Share This Post', 'width=640,height=450');return false" class="btn btn-share share facebook" aria-label="share facebook"> <i class="icon-facebook"></i>  <span>Facebook</span> </button> <button type="button" onclick="window.open('https://twitter.com/intent/tweet?url=https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only&text=Audit+Title+and+Meta+Description+Lengths+from+a+Sitemap+with+Python+%28Standard+Library+Only%29', 'Share This Post', 'width=640,height=450');return false" class="btn btn-share share twitter" aria-label="share x"> <i class="icon-twitter"></i>  <span>Twitter</span> </button> <a href="https://api.whatsapp.com/send?text=Audit Title and Meta Description Lengths from a Sitemap with Python (Standard Library Only) - https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only" target="_blank" class="btn btn-share share whatsapp"> <i class="icon-whatsapp"></i>  <span>Whatsapp</span> </a> <button type="button" onclick="window.open('http://www.linkedin.com/shareArticle?mini=true&url=https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only', 'Share This Post', 'width=640,height=450');return false" class="btn btn-share share linkedin" aria-label="share linkedin"> <i class="icon-linkedin"></i>  <span>Linkedin</span> </button> <button type="button" onclick="window.open('http://pinterest.com/pin/create/button/?url=https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only&media=https://media2.dev.to/dynamic/image/width%3D1200,height%3D627,fit%3Dcover,gravity%3Dauto,format%3Dauto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcqslpq2mjsmse85duwnz.png', 'Share This Post', 'width=640,height=450');return false" class="btn btn-share share pinterest" aria-label="share pinterest"> <i class="icon-pinterest"></i>  <span>Pinterest</span> </button> </div> </div> <div class="row"> </div> </div> <div class="row"> <div class="col-12"> <div class="d-flex about-author"> <div class="flex-shrink-0"> <a href="https://phu.lt/en/profile/phult"> <img src="https://phu.lt/uploads/profile/avatar_1_648962eef2ca0.jpg" alt="phult" class="img-fluid rounded-1" width="110" height="110"> </a> </div> <div class="flex-grow-1 ms-3"> <p class="fw-bold mb-2"> <a href="https://phu.lt/en/profile/phult"> phult </a> </p> <p class="mb-2"> Phu.lt chia sẻ kiến thức về công nghệ và lập trình, từ phát triển phần mềm, DevOps, Linux và kiến trúc hệ thống đến những diễn biến mới trong khoa học và công nghệ. Mỗi bài viết hướng đến việc giải thích rõ cách công nghệ hoạt động, giá trị thực tế và những giới hạn cần biết. </p> <ul class="d-flex align-items-center flex-wrap gap-2 author-social"> <li><a class="facebook" href="https://www.facebook.com/phu.ltd" target="_blank"><i class="icon-facebook"></i></a></li> <li><a class="twitter" href="https://twitter.com/trieuphule" target="_blank"><i class="icon-twitter"></i></a></li> <li><a class="linkedin" href="https://www.linkedin.com/in/phultd/" target="_blank"><i class="icon-linkedin"></i></a></li> </ul> </div> </div> </div> </div> <div class="related-posts"> <div class="widget-title"> <h4 class="title">Related Posts</h4> </div> <div class="widget-body"> <div class="row post-list"> <div class="col-sm-12 col-md-6 col-lg-4 mb-3"> <div class="position-relative mb-3"> <a href="https://phu.lt/en/xai-founder-musk-grok-bot-advances-efficiency-and-behavioral-improvements"> <div class="post-image-container"> <img data-src="https://www.techflowpost.com/favicon/twitter-image.png" class="img-fluid lazyload" alt="xAI Founder Musk: Grok Bot Advances Efficiency and Behavioral Improvements" width="270" height="180"> </div> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/xai-founder-musk-grok-bot-advances-efficiency-and-behavioral-improvements" class="d-block">xAI Founder Musk: Grok Bot Advances Efficiency and Behavioral Imp...</a> </h3> </div> <div class="col-sm-12 col-md-6 col-lg-4 mb-3"> <div class="position-relative mb-3"> <a href="https://phu.lt/en/when-the-platform-suspends-you-two-sides-of-it-wasnt-for-nothing"> <div class="post-image-container"> <img data-src="https://media2.dev.to/dynamic/image/width%3D1200,height%3D627,fit%3Dcover,gravity%3Dauto,format%3Dauto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flpp1ijt7kqkwax5zbdb5.png" class="img-fluid lazyload" alt="When the platform suspends you: two sides of 'it wasn't for nothing'" width="270" height="180"> </div> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/when-the-platform-suspends-you-two-sides-of-it-wasnt-for-nothing" class="d-block">When the platform suspends you: two sides of 'it wasn't for nothi...</a> </h3> </div> <div class="col-sm-12 col-md-6 col-lg-4 mb-3"> <div class="position-relative mb-3"> <a href="https://phu.lt/en/google-cloud-adds-self-managed-vmware-cloud-foundation-on-bare-metal"> <div class="post-image-container"> <img data-src="https://res.infoq.com/news/2026/11/google-cloud-self-managed-vmware/en/headerimage/generatedHeaderImage-1793604325753.jpg" class="img-fluid lazyload" alt="Google Cloud Adds Self-Managed VMware Cloud Foundation on Bare Metal" width="270" height="180"> </div> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/google-cloud-adds-self-managed-vmware-cloud-foundation-on-bare-metal" class="d-block">Google Cloud Adds Self-Managed VMware Cloud Foundation on Bare Me...</a> </h3> </div> </div> </div> </div> <div class="row"> <div class="col-12"> <div class="comments-section"> </div> </div> </div> </div> <div class="col-md-12 col-lg-4"> <div class="sticky-lg-top"> <div class="row"> <div class="col-12"> <div class="container-bn container-bn-ds m-b-30"> <div class="bn-content bn-sidebar-content"> <div class="bn-inner bn-ds-20"> <a href="https://sedo.com/?language=us&campaignId=331719" target="_parent"> <img src="https://cdn.sedo.com/c7r/assets/static/images/banner/en/Sedo_Banner_336280_03.jpg" width="336" height="280"> </a> </div> </div> </div> </div> <div class="col-12 sidebar-widget widget-popular-posts"> <div class="widget-title"> <h4 class="title">Popular Posts</h4> </div> <div class="widget-body"> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/binance-wallet-predict-alpha-trading-launch-has-been-postponed-the-specific-time-will-be-announced-separately" aria-label="Binance Wallet: PREDICT Alpha trading launch has been postponed. The specific time will be announced separately."> <div class="post-image-container"> <img data-src="https://www.techflowpost.com/favicon/twitter-image.png" class="img-fluid lazyload" alt="Binance Wallet: PREDICT Alpha trading launch has been postponed. The specific time will be announced separately." width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/binance-wallet-predict-alpha-trading-launch-has-been-postponed-the-specific-time-will-be-announced-separately" class="d-block">Binance Wallet: PREDICT Alpha trading launch has been postponed. ...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Oct 30, 2026</span> </div> </div> </div> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/8-emudhra-certificates-are-compromised-heres-what-you-need-to-know" aria-label="8 eMudhra certificates are compromised. Here's what you need to know"> <div class="post-image-container"> <img data-src="https://blog.cloudflare.com/_emdash/api/media/file/01M5DK5FY53PEFH46MT990YMSR.png?#" class="img-fluid lazyload" alt="8 eMudhra certificates are compromised. Here's what you need to know" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/8-emudhra-certificates-are-compromised-heres-what-you-need-to-know" class="d-block">8 eMudhra certificates are compromised. Here's what you need to k...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Oct 21, 2026</span> </div> </div> </div> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/spacex-and-tesla-boards-form-special-committees-to-review-merger" aria-label="SpaceX and Tesla boards form special committees to review merger"> <div class="post-image-container"> <img data-src="https://techcrunch.com/wp-content/uploads/2026/10/space-tesla-cybertruck.jpg?resize=1200,801" class="img-fluid lazyload" alt="SpaceX and Tesla boards form special committees to review merger" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/spacex-and-tesla-boards-form-special-committees-to-review-merger" class="d-block">SpaceX and Tesla boards form special committees to review merger</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Oct 21, 2026</span> </div> </div> </div> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/hugging-face-releases-carbon-x-an-open-source-14b-parameter-multimodal-ai-model-to-let-users-query-dna-sequences-via-natural-language-trained-on-300b-tokens-emily-mullinwired" aria-label="Hugging Face releases Carbon-X, an open-source, 14B-parameter multimodal AI model to let users query DNA sequences via natural language, trained on 300B tokens (Emily Mullin/Wired)"> <div class="post-image-container"> <img data-src="https://media.wired.com/photos/6ad92806649bdf60df6fd54f/191:100/w_1280,c_limit/Science_Hugging%20Face%E2%80%99s%20Powerful%20New%20DNA%20Model%20Is%20Small%20Enough%20to%20Run%20on%20a%20Laptop_v1.jpg" class="img-fluid lazyload" alt="Hugging Face releases Carbon-X, an open-source, 14B-parameter multimodal AI model to let users query DNA sequences via natural language, trained on 300B tokens (Emily Mullin/Wired)" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/hugging-face-releases-carbon-x-an-open-source-14b-parameter-multimodal-ai-model-to-let-users-query-dna-sequences-via-natural-language-trained-on-300b-tokens-emily-mullinwired" class="d-block">Hugging Face releases Carbon-X, an open-source, 14B-parameter mul...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Oct 22, 2026</span> </div> </div> </div> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/3-of-10-colors-failed-safaris-dark-mode-contrast-bar" aria-label="3 of 10 colors failed Safari's dark-mode contrast bar"> <div class="post-image-container"> <img data-src="https://media2.dev.to/dynamic/image/width%3D1200,height%3D627,fit%3Dcover,gravity%3Dauto,format%3Dauto/https:%2F%2Fraw.githubusercontent.com%2Ffrankchu91%2FWhatTechPost%2Fmain%2Fpublished%2F2026-11-03-safari-dark-mode%2Fcover.png" class="img-fluid lazyload" alt="3 of 10 colors failed Safari's dark-mode contrast bar" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/3-of-10-colors-failed-safaris-dark-mode-contrast-bar" class="d-block">3 of 10 colors failed Safari's dark-mode contrast bar</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 4, 2026</span> </div> </div> </div> </div> </div> <div class="col-12 sidebar-widget"> <div class="widget-title"> <h4 class="title">Categories</h4> </div> <div class="widget-body"> <div class="d-flex flex-wrap sidebar-categories"> <a href="https://phu.lt/en/tech-news" class="btn"> Tech news  <span class="badge text-bg-secondary">3380</span> </a> </div> </div> </div> <div class="col-12 sidebar-widget"> <div class="widget-title"> <h4 class="title">Random Posts</h4> </div> <div class="widget-body"> <div class="single-slider-container random-slider-container"> <div id="randomSlider" class="single-slider random-slider"> <div class="slider-item position-relative"> <a href="https://phu.lt/en/swap-one-seam-dual-run-both-implementations-then-delete-the-old-path" aria-label="Swap One Seam, Dual-Run Both Implementations, Then Delete the Old Path"> <div class="post-image-container post-image-slider"> <img data-lazy="https://media2.dev.to/dynamic/image/width%3D1200,height%3D627,fit%3Dcover,gravity%3Dauto,format%3Dauto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcuha2t6msi32fpdd6ygq.png" class="img-fluid" alt="Swap One Seam, Dual-Run Both Implementations, Then Delete the Old Path" width="412" height="278"> </div> </a> <div class="caption redirect-onclik" data-url="https://phu.lt/en/swap-one-seam-dual-run-both-implementations-then-delete-the-old-path"> <div class="d-flex position-relative justify-content-start"> <a href="https://phu.lt/en/tech-news"> <span class="label-post-category">Tech news</span> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/swap-one-seam-dual-run-both-implementations-then-delete-the-old-path">Swap One Seam, Dual-Run Both Implementations, Then Delete the Old...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 7, 2026</span> </div> </div> </div> <div class="slider-item position-relative"> <a href="https://phu.lt/en/from-the-front-row-to-the-hot-seat-what-the-midterms-mean-for-big-tech" aria-label="From the front row to the hot seat — what the midterms mean for Big Tech"> <div class="post-image-container post-image-slider"> <img data-lazy="https://techcrunch.com/wp-content/uploads/2026/11/SHAWN-THEW-Contributor-Getty.jpg?w=1024" class="img-fluid" alt="From the front row to the hot seat — what the midterms mean for Big Tech" width="412" height="278"> </div> </a> <div class="caption redirect-onclik" data-url="https://phu.lt/en/from-the-front-row-to-the-hot-seat-what-the-midterms-mean-for-big-tech"> <div class="d-flex position-relative justify-content-start"> <a href="https://phu.lt/en/tech-news"> <span class="label-post-category">Tech news</span> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/from-the-front-row-to-the-hot-seat-what-the-midterms-mean-for-big-tech">From the front row to the hot seat — what the midterms mean for B...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 7, 2026</span> </div> </div> </div> <div class="slider-item position-relative"> <a href="https://phu.lt/en/trump-roundup-the-past-24-hours" aria-label="Trump Roundup: The Past 24 Hours"> <div class="post-image-container post-image-slider"> <img data-lazy="https://www.techflowpost.com/favicon/twitter-image.png" class="img-fluid" alt="Trump Roundup: The Past 24 Hours" width="412" height="278"> </div> </a> <div class="caption redirect-onclik" data-url="https://phu.lt/en/trump-roundup-the-past-24-hours"> <div class="d-flex position-relative justify-content-start"> <a href="https://phu.lt/en/tech-news"> <span class="label-post-category">Tech news</span> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/trump-roundup-the-past-24-hours">Trump Roundup: The Past 24 Hours</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 7, 2026</span> </div> </div> </div> <div class="slider-item position-relative"> <a href="https://phu.lt/en/data-centers-are-being-powered-by-giant-gas-plants-possibly-increasing-carbon-pollution-for-decades" aria-label="Data Centers are Being Powered By Giant Gas Plants, Possibly Increasing Carbon Pollution for Decades"> <div class="post-image-container post-image-slider"> <img data-lazy="https://a.fsdn.com/sd/twitter_icon_large.png" class="img-fluid" alt="Data Centers are Being Powered By Giant Gas Plants, Possibly Increasing Carbon Pollution for Decades" width="412" height="278"> </div> </a> <div class="caption redirect-onclik" data-url="https://phu.lt/en/data-centers-are-being-powered-by-giant-gas-plants-possibly-increasing-carbon-pollution-for-decades"> <div class="d-flex position-relative justify-content-start"> <a href="https://phu.lt/en/tech-news"> <span class="label-post-category">Tech news</span> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/data-centers-are-being-powered-by-giant-gas-plants-possibly-increasing-carbon-pollution-for-decades">Data Centers are Being Powered By Giant Gas Plants, Possibly Incr...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 8, 2026</span> </div> </div> </div> <div class="slider-item position-relative"> <a href="https://phu.lt/en/metr-anthropic-models-are-advancing-at-roughly-twice-the-pace-of-a-year-ago" aria-label="METR: Anthropic Models Are Advancing at Roughly Twice the Pace of a Year Ago"> <div class="post-image-container post-image-slider"> <img data-lazy="https://www.techflowpost.com/favicon/twitter-image.png" class="img-fluid" alt="METR: Anthropic Models Are Advancing at Roughly Twice the Pace of a Year Ago" width="412" height="278"> </div> </a> <div class="caption redirect-onclik" data-url="https://phu.lt/en/metr-anthropic-models-are-advancing-at-roughly-twice-the-pace-of-a-year-ago"> <div class="d-flex position-relative justify-content-start"> <a href="https://phu.lt/en/tech-news"> <span class="label-post-category">Tech news</span> </a> </div> <h3 class="title"> <a href="https://phu.lt/en/metr-anthropic-models-are-advancing-at-roughly-twice-the-pace-of-a-year-ago">METR: Anthropic Models Are Advancing at Roughly Twice the Pace of...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 7, 2026</span> </div> </div> </div> </div> </div> </div> </div> <div class="col-12 sidebar-widget"> <div class="widget-title"> <h4 class="title">Popular Tags</h4> </div> <div class="widget-body"> <ul class="d-flex flex-wrap popular-tags"> <li><a href="https://phu.lt/en/tag/ai-agent">AI agent</a></li> <li><a href="https://phu.lt/en/tag/tri-tue-nhan-tao">trí tuệ nhân tạo</a></li> <li><a href="https://phu.lt/en/tag/an-ninh-mang">an ninh mạng</a></li> <li><a href="https://phu.lt/en/tag/robotics">robotics</a></li> <li><a href="https://phu.lt/en/tag/quantum-computing">quantum computing</a></li> <li><a href="https://phu.lt/en/tag/an-toan-ai">an toàn AI</a></li> <li><a href="https://phu.lt/en/tag/agentic-ai">Agentic AI</a></li> <li><a href="https://phu.lt/en/tag/may-tinh-luong-tu">máy tính lượng tử</a></li> <li><a href="https://phu.lt/en/tag/cybersecurity">cybersecurity</a></li> <li><a href="https://phu.lt/en/tag/openai">OpenAI</a></li> <li><a href="https://phu.lt/en/tag/mit">MIT</a></li> <li><a href="https://phu.lt/en/tag/ai">AI</a></li> <li><a href="https://phu.lt/en/tag/embodied-ai">Embodied AI</a></li> <li><a href="https://phu.lt/en/tag/physical-ai">Physical AI</a></li> <li><a href="https://phu.lt/en/tag/ai-cybersecurity">AI cybersecurity</a></li> </ul> </div> </div> <div class="col-12 sidebar-widget"> </div> </div> </div> </div> </div> </div> </section> <style> .post-text img { display: none !important; } .post-content .post-summary { display: none !important; } </style> <script type="application/ld+json">{"@context":"https://schema.org","@type":"Organization","@id":"https://phu.lt/#organization","name":"Infinite","url":"https://phu.lt/","description":"Infinite - Blog Magazine Script","logo":{"@type":"ImageObject","url":"https://phu.lt/uploads/logo/logo_6a6396b888f130-39409549.png","contentUrl":"https://phu.lt/uploads/logo/logo_6a6396b888f130-39409549.png","width":192,"height":45}}</script><script type="application/ld+json">{"@context":"https://schema.org","@type":"WebSite","@id":"https://phu.lt/#website","name":"Infinite - Blog Magazine Script","url":"https://phu.lt/","description":"Infinite - Blog Magazine Script","inLanguage":"en-US","publisher":{"@id":"https://phu.lt/#organization"},"potentialAction":{"@type":"SearchAction","target":"https://phu.lt/en/search?q={search_term_string}","query-input":"required name=search_term_string"}}</script><script type="application/ld+json">{"@context":"https://schema.org","@type":"NewsArticle","@id":"https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only#article","mainEntityOfPage":{"@type":"WebPage","@id":"https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only"},"url":"https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only","headline":"Audit Title and Meta Description Lengths from a Sitemap with Python (Standard Library Only)","name":"Audit Title and Meta Description Lengths from a Sitemap with Python (Standard Library Only)","description":"A sitemap can be valid XML and still ship titles like Home or meta descriptions that are empty, identical across fifty product pages, or long enough to get cut mid-sentence in a SERP. None of that shows up as a sitemap error. You only see it when you open each page, or when Search Console shows a title Google rewrote for you.\n\nThis post builds a small auditor in Python using only the standard library (html.parser, urllib, xml.etree, csv, concurrent.futures). It reads a sitemap or sitemap index, fetches every listed URL, extracts the and meta name=\u0022description\u0022, measures their character lengths, flags missing and empty values, and marks duplicate titles or descriptions across the set. The output is a CSV you can sort in a spreadsheet.\n\nGoogle Search Central docs on title links and snippets / meta descriptions were opened on 7 Nov 2026. Appie free-tool descriptions are as of 7 Nov 2026. The script was tested on Python 3.13.5 against a local fixture site, shown below.\n\nDisclosure: Appie.","articleSection":"Tech news","datePublished":"2026-11-08T23:00:02+07:00","dateModified":"2026-11-08T23:00:02+07:00","inLanguage":"en-US","author":{"@type":"Organization","@id":"https://phu.lt/#organization","name":"Infinite","url":"https://phu.lt/"},"publisher":{"@type":"Organization","@id":"https://phu.lt/#organization","name":"Infinite","url":"https://phu.lt/","logo":{"@type":"ImageObject","url":"https://phu.lt/uploads/logo/logo_6a6396b888f130-39409549.png","contentUrl":"https://phu.lt/uploads/logo/logo_6a6396b888f130-39409549.png","width":192,"height":45}},"isAccessibleForFree":true,"image":["https://media2.dev.to/dynamic/image/width%3D1200,height%3D627,fit%3Dcover,gravity%3Dauto,format%3Dauto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fcqslpq2mjsmse85duwnz.png"],"keywords":["Audit","Title","and","Meta","Description","Lengths","from","Sitemap","with","Python","Standard","Library","Only"]}</script><script type="application/ld+json">{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://phu.lt/en"},{"@type":"ListItem","position":2,"name":"Tech news","item":"https://phu.lt/en/tech-news"},{"@type":"ListItem","position":3,"name":"Audit Title and Meta Description Lengths from a Sitemap with Python (Standard Library Only)","item":"https://phu.lt/en/audit-title-and-meta-description-lengths-from-a-sitemap-with-python-standard-library-only"}]}</script> <footer id="footer"> <div class="container-xl"> <div class="row footer-content"> <div class="col-md-12 col-lg-4"> <div class="footer-widget"> <h4 class="title">About</h4> <div class="title-line"></div> <p></p> </div> </div> <div class="col-md-12 col-lg-4"> <div class="footer-widget"> <h4 class="title">Latest Posts</h4> <div class="title-line"></div> <div class="footer-posts"> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/cloudflare-scales-d1-control-plane-to-millions-of-databases" aria-label="Cloudflare Scales D1 Control Plane to Millions of Databases"> <div class="post-image-container"> <img data-src="https://res.infoq.com/news/2026/11/cloudflare-d1-control-plane/en/headerimage/generatedHeaderImage-1792577891694.jpg" class="img-fluid lazyload" alt="Cloudflare Scales D1 Control Plane to Millions of Databases" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/cloudflare-scales-d1-control-plane-to-millions-of-databases" class="d-block">Cloudflare Scales D1 Control Plane to Millions of Databases</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 8, 2026</span> </div> </div> </div> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/google-cloud-adds-self-managed-vmware-cloud-foundation-on-bare-metal" aria-label="Google Cloud Adds Self-Managed VMware Cloud Foundation on Bare Metal"> <div class="post-image-container"> <img data-src="https://res.infoq.com/news/2026/11/google-cloud-self-managed-vmware/en/headerimage/generatedHeaderImage-1793604325753.jpg" class="img-fluid lazyload" alt="Google Cloud Adds Self-Managed VMware Cloud Foundation on Bare Metal" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/google-cloud-adds-self-managed-vmware-cloud-foundation-on-bare-metal" class="d-block">Google Cloud Adds Self-Managed VMware Cloud Foundation on Bare Me...</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 8, 2026</span> </div> </div> </div> <div class="d-flex post-item-small"> <div class="col-left flex-shrink-0"> <a href="https://phu.lt/en/entires-git-forge-rethinks-git-collaboration-for-coding-agents" aria-label="Entire's Git Forge Rethinks Git Collaboration for Coding Agents"> <div class="post-image-container"> <img data-src="https://res.infoq.com/news/2026/11/entire-git-forge-coding-agents/en/headerimage/entire-git-forge-agents-1794134163813.jpeg" class="img-fluid lazyload" alt="Entire's Git Forge Rethinks Git Collaboration for Coding Agents" width="130" height="90"> </div> </a> </div> <div class="flex-grow-1"> <h3 class="title"> <a href="https://phu.lt/en/entires-git-forge-rethinks-git-collaboration-for-coding-agents" class="d-block">Entire's Git Forge Rethinks Git Collaboration for Coding Agents</a> </h3> <div class="d-flex align-items-center flex-wrap post-meta"> <a href="https://phu.lt/en/profile/phult" class="fw-semibold">phult</a> <span><i class="icon-clock"></i> Nov 8, 2026</span> </div> </div> </div> </div> </div> </div> <div class="col-md-12 col-lg-4"> <div class="row"> <div class="col-12"> <div class="footer-widget widget-follow"> <h4 class="title">Social Media</h4> <div class="title-line"></div> <ul> <li><a class="rss" href="https://phu.lt/en/rss-feeds" aria-label="rss"><i class="icon-rss"></i></a></li> </ul> </div> </div> <div class="col-12"> </div> </div> </div> </div> <div class="footer-bottom"> <div class="d-flex justify-content-between flex-wrap"> <div class="flext-item copyright"> Copyright 2022 Infinite - All Rights Reserved. </div> <div class="flext-item"> <ul class="nav-footer"> <li><a href="https://phu.lt/en/terms-conditions">Terms & Conditions </a></li> </ul> </div> </div> </div> </div> </footer> <div class="cookies-warning"> <button type="button" aria-label="close cookies warning" class="close" onclick="closeCookiesWarning();"> <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="bi bi-x" viewBox="0 0 16 16"> <path d="M4.646 4.646a.5.5 0 0 1 .708 0L8 7.293l2.646-2.647a.5.5 0 0 1 .708.708L8.707 8l2.647 2.646a.5.5 0 0 1-.708.708L8 8.707l-2.646 2.647a.5.5 0 0 1-.708-.708L7.293 8 4.646 5.354a.5.5 0 0 1 0-.708z"/> </svg> </button> <div class="text"> <p>This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies <a href="#">Find out more here</a></p> </div> <button type="button" class="btn btn-default" onclick="closeCookiesWarning();">Accept Cookies</button> </div> <a href="#" class="scrollup"><i class="icon-arrow-up"></i></a> <script src="https://phu.lt/assets/js/jquery-3.7.1.min.js"></script> <script src="https://phu.lt/assets/vendor/bootstrap/bootstrap.bundle.min.js"></script> <script src="https://phu.lt/assets/js/plugins-4.5.js"></script> <script src="https://phu.lt/assets/js/script-4.5.min.js"></script> <script>$('<input>').attr({type: 'hidden', name: 'lang', value: InfConfig.sysLangId}).appendTo('form');</script> <!-- Google tag (gtag.js) --> <script async src="https://www.googletagmanager.com/gtag/js?id=G-12FTV5TFZ4"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'G-12FTV5TFZ4'); </script><!-- Google tag (gtag.js) --> <script async src="https://www.googletagmanager.com/gtag/js?id=G-12FTV5TFZ4"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'G-12FTV5TFZ4'); </script> <script src="https://updatly.live/embed/site_68c0e5277d460" defer></script> </body> </html>