Ruff fixes, other minor improvements
- mistat-fake-data-gen.py+1-3
- mistat-tests.py+1217-504
- mistat.py+584-320
Fix RSS description processing and fallback logic
Implemented lazy loading and tag cloud
Many new updates, fixes everywhere
Added more robust and vesatile `mistat-fake-data-gen.sh` replacement
- mistat-fake-data-gen.py+206
Refactor: Simplify script, use Unix timestamps, add categories & year dirs
This commit introduces a major refactoring of the mistat script, simplifying its structure and core logic while retaining essential features and adding new ones based on recent requirements.
Key changes include:
- **Core Refactoring:** Replaced the previous complex script with a streamlined Python version focusing on core blogging features.
- **Filename Change:** Posts now use Unix timestamps as filenames (e.g., `1745828957.log`) instead of the `YYYY-MM-DD-HH-MM-SS` format.
- **Directory Structure:** Output HTML files for posts are now organized into year-based directories (e.g., `public/2024/1745828957.html`).
- **Simplified Categories:** Category detection now relies on a single hashtag placed on the last line of the source file.
- **Category & Pagination Pages:** Added generation of paginated index pages for each category found. Directory names are configurable (`c/`, `p/` by default).
- **Relative Links:** Implemented relative path generation for internal navigation links (Home, RSS, Pagination, Category links) to allow local file browsing.
- **Directory Redirects:** Added simple `index.html` redirect pages in year, category, and pagination subdirectories.
- **Markdown Processing:** Uses a robust internal processor for explicit links (`[text](url)`), angle-bracket autolinks (`<url>`), bold, italics, inline code, and code blocks. Bare URL autolinking is disabled.
- **Previews:** Implemented previews for titled posts on list pages, showing only the first block with a "Read More" indicator.
- **UI Tweaks:** Added optional root backlink, site description on index pages, and a CSS class (`titled-post`) for titled articles on list views.
- **Features Kept:** RSS feed generation, incremental builds (`generate-diff`), CSP nonce for inline CSS.
- **Features Removed:** Dropped numerous features from the original script, including translations, advanced linklog handling, themes, separate contact/about/stats pages, XLS export, etc., to achieve simplicity.
- README.md+47-64
- index.php-1003
- mistat-cli-tests.py-150
- mistat-cli.py-3467
- mistat-fake-data-gen.sh-48
- mistat.py+672
Minor fixes to VIDEO_URL_PATTERN
Implemented Dailymotion parsing and embeding
* added visible labels to progress, meter, and checkbox elements for improved screen reader (and human) support
* introduced `COMPLETED_TRANSLATION` and `PENDING_TRANSLATION` for checkbox labels
* disabled checkbox interaction using `pointer-events: none;` due to smaller label size
This commit publishes mistat-cli.py v0.9.3, introducing several new features and improvements:
* **Video Embedding:** Re-enabled Vimeo video embedding, including:
* Optional Content Security Policy (CSP) integration
* Customized cookie banner for Vimeo
* **Enhanced Markdown Parsing:** Added support for:
* `<progress>` elements (`[progress:0-100]`)
* `<meter>` elements (`[meter:0-100:Optional text]`)
* Checkbox parsing (`[x]` and `[ ]`)
* Improved accessibility with unique, randomly generated IDs
* **Checkbox Styling:** Added CSS for `input type="checkbox"` with:
* `pointer-events: none;` and `user-select: none;` to prevent accidental checkbox interactions.
* Note: Further styling for the `disabled` state is planned.
* **Article Preview Generation:** Improved the `generate_preview()` function for articles.
* **Translation Refinements:** Made minor improvements to translations.
* **SVG Video Icon:** Added a new SVG video icon from the Feather icon set.
mistat-cli.py v0.9.1 - A Thousand Cuts Edition (like in Jade Empire)
- Refined recently implemented title recognition for multi-line files with heading (starting with `#`).
- Improved HTML and RSS output; RSS now displays article titles, enabling basic blog functionality.
- Deprecated `USE_STANDARD_BLOG_HEADERS_ON_INDIVIDUAL_POSTS` and `SHOW_HEADERS_FOR_POSTS_LONGER_THAN_THIS_NUMBER_OF_CHARACTERS`.
- Enabled HTML/XML minification by default.
- Increased default number of XML posts to 25.
- Disabled `BLOCK_ROBOTS_USING_HTML_METATAGS` and `BLOCK_AI_USING_HTML_METATAGS` by default.
- Updated `AVERAGE_PAGE_SIZE_KB` to 2374 KB (source: https://almanac.httparchive.org).
- Added `YEAR_ONLY` timestamp format.
- Minor adjustments to emojis and HTML entity characters.
- CSS enhancements, including the addition of `nowrap` in several locations.
- Disabled "Potential emissions reduction" HTML comment; future simplification planned.
- Improved `generate_preview` function to handle `<ul>` and `<pre>` elements.
- Numerous other HTML tweaks and minor improvements.
CSS edits, centered paginator at the top
Tagging `0.9.0` Release:
After several years of development, version `0.9.0` marks a significant step toward stabilization and optimization ahead of the `1.0.0` release. While version `0.7.0` already offered a clean and simple foundation, version `0.8.0` introduced numerous features, adding complexity. The changes in this release include:
- Added support for reading externalized configurations, enabling use of the script for various purposes, such as tracking blog status, link logs, and more. This is already being used for 2 projects, which means the project is now powering 2 websites, leading to a 50% increase in usage 😄
- Introduced three visual (UX?) display modes: (almost) text-only, emoji-based, and SVG view. Users can now choose whether to display key UI elements and decide between emojis (browser support still evolving, relying on `font-variant-emoji: text`) or SVG icons from the Feather icon set.
- Expanded the SVG icon set, including improvements to accessibility (e.g., ARIA labels).
- Added embedded SVG icons for external links and `<time>` elements in CSS to optimize timestamp display and minimize unnecessary code.
- Replaced the broken flex-based CSS vertical alignment with a simpler top padding approach using `clamp()` to avoid scrollbars on mobile devices.
Though the list may seem modest, considerable effort was put into reaching this point. Feature complete, only "minor" optimizations remain before reaching `1.0.0` 🙃
Replaced `★` with emoji star (⭐) for special bookmarked entries, for testing. Should be better and more vivid than HTML entity char.
Fix URL matching for cases ending with ')'. Also, allowed single quotes in URLs but prioritize sanitization over supporting all characters.
Added `translate="no"` to links to prevent link strings being translated
Updated `svg_anchor_back_to_top_icon` fill to white to improve visibility overall
Updated URL regex - still have an edge case for URLs ending with ")"
Check if the message is longer than 1 character and if not, abort exec
mistat-cli-tests.py Unit Tests 0.2.0
- Added tracking for special pages (linklog, contact, statistics)
- Made search for `date` optional, with a `self.assertGreater` message indicating where no dates were found. It is no longer a critical error but helpful to know.
- Added search for empty single ('') and double quotes (""). This was particularly valuable during the transition to a simpler template and changes to the underlying CSS style.
- mistat-cli-tests.py+33-10
Mistat 0.8.72
- Added `translate="no"` to project name in <h1> to prevent translation by search engines (see: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/translate)
- Revamped CSS to be simpler and more efficient, achieving a ~30% reduction in size. It is now more minimalistic but faster and easier to maintain.
- Various tweaks and smaller fixes across the board. Added `translate="no"` on the contact page too, to prevent translation of email field (though Firefox AI translation does not yet respect this attribute).
Minor tweaks for `url_referrerpolicy_external` on linklog page
Allow removing linklog entry limits by using a falsy value (0 or None) for NUMBER_OF_LINKLOG_ENTRIES_ON_STANDALONE_LINKLOG_PAGE. When falsy, the limit is removed, and `∞` is displayed to indicate no upper limit (e.g., 36 / ∞)
Initial commit of mistat-cli-tests.py - v0.1.0
- Includes tests for HTML generation, source-to-HTML mapping, RSS feed validation, and metadata presence
- Implements checks for date-based HTML files, non-empty HTML files, and UTF-8 encoding
- Optimized file reading using caching for better performance with large datasets
Chrome added support for `font-variant-emoji` CSS property in version 131 (November 2024) meaning i can keep `font-variant-emoji: text;` for everything and use `font-variant-emoji: normal;` for article body!
Cookie notices on pages with iframe videos are red now. Disabled special icons on individual posts for now because they do look like actionable/clickable icons. Fixed navigation at the bottom on index pages: homepage icon is not generated anymore on index.html, only index-XYZ.html
- README.md+1-1
- mistat-cli.py+75-10
Mistat 0.8.66
New:
- introduced option to change timestamps of created files, with user defined offset
- improved link regex and introduced new way of forming (s)html/php links
Fixes:
- various translation fixes
- disabled COOKIE_NOTICE_TRANSLATION sanitizing to prevent breaking html code in it (YT cookie policy link)
Removed:
- removed automatic creation of local links to posts: user can control that flow now
Tagged 0.8.65. Changes:
- variables sanitization! We are checking most stuff now
- added "About page", design differs a bit - contact icon is enlarged (if it is used), user can add markdown text
- fixed a few bugs left and right
0.8.64 changes:
- introduced variable checker and setter, for safer env
- introduced translation for #top, #content and #footer anchors on the page
- further reduced CSS - we are not inserting anchoring code anymore on each page, that is 2 classes less per page (i know, i know)
- reduced spacing around linklog entries on index pages
- removed old comments, no need to keep them in code
Bumping to 0.8.62
Minor fixes to CSS conditionals + enabled top anchor arrow on linklog page
Warping to v0.8.61 with some interesting changes:
- Added a linklog page and made several refinements to it. Unfortunately, it is only generated when you use `generate-all` to regenerate all posts. This is a limitation of the flat-file system; to update linklogs that might be scattered, we need to scan all posts. However, I'm pleased with the outcome and accept this tradeoff.
- Enhanced the standalone contact page with better CSS support and more randomly generated content for users who choose to hide their real email. While improvements have been made, it’s not completely secure, so creating a separate email for website contact remains essential for the foreseeable future.
- Added a "Jump to content/footer" link to improve accessibility. I've been testing this on Opera Mini 4.4, and it seems to work well. More work will follow after I begin using a screen reader.
- Made improvements to microformats after various tests.
- Made minor HTML tweaks to reduce overall page size and decrease the number of CSS classes.
- Updated the YouTube regex to capture @profiles.
- Made minor adjustments to SVG descriptions for better compatibility with browsers lacking strong SVG support (like NetSurf and w3).
- Introduced simple CSS dynamics. Certain standalone pages will now have slightly different CSS. For example, the contact page's CSS won't appear on other pages, saving 0.3 KB per page.
- Reduced overall CSS complexity by using a "CSS Code Quality calculator," which was quite helpful.
- Dark theme has returned; we no longer support a light theme.
- Added an option to shuffle the accent color on the blog with each rebuild.
Fixed some minor bugs:
- Added an anchor to the body; I’m not sure how I missed this, but the #top anchor in browsers now returns to the top of the page by default.
- Reduced the number of source file listings. Previously, we had several instances listing source files, but that is now streamlined. Testing with 20,000 files showed positive results as we now only access the filesystem once.
Other numerous improvements I can’t recall at the moment because I haven't kept track like a real software developer!
Many exciting changes:
- added option to embed YT videos on page. This option transforms youtube.com links to youtube-nocookie.com, adds message bellow each iframe about potential cookes (and into RSS for entries with iframes)
- added post video badge for all posts with YouTube videos. SVG icon used is, again, from Feather set (https://github.com/feathericons/feather) which i optimized via SVGOMG (https://jakearchibald.github.io/svgomg/).
- added HTML comments about emmision savings (https://sustainablewebdesign.org/estimating-digital-emissions/) and average page size (https://httparchive.org/reports). It is a nice stat but hidden from users. Also, i have added `iframe` counter per generated page. Because i can
Fixes:
- improved MARKED_PATTERN regex
- reverted back CSS style to white, made many MANY CSS improvements
- improved post tagging since we got new a tag now. Right now we have special posts (bookmarks), video posts, linklog
- little tunes and tweaks here and there
Note: video embeding is prepared and working for Vimeo too however vimeo sets cookies before it even loads video meaning that it created headaches with GDPR.
Plenty of changes:
- profanity filter!
- option to post shorter link only statuses (linkblog)
- added translation strings
- expanded timestamp formats
- improved cache buster strings
- many regex improvements
- moved some SVG icons to css to optimize rendering and html pagesize
- many small fixes and tweaks to CSS and code
New modules:
- ulrlib.parse (unquote)
DEV mistat-cli.py:
- added way to calculate RSS update frequency
- fixed minor issues with website card (they use full URL now, fixing some issues with WhatsApp and probably other platforms)
- fixed titles
- added RSS update frequency
- mayor CSS refactoring, changed logic and allowed CSS externalization (CSP loves this and makes my life easier too)
- added option to input a link for Mastodon verification
- added robots 'noindex,nofollow' to main page, to avoid indexing
- many other improvements
mproved mistat-cli.py left and right:
- removed `htmlmin` and started using our function now to reduce ext. dependencies
- added a way to get TZ from local machine without using any additional modules
- introduced base CSS building blocks, reused between my other projects
- removed dark theme (WCAG AAA and dark theme != safe eyes)
- added option to linkinfy timestamps within articles. It can be used as a permalink system to link articles using only filename
- renamed RSS file from feed.xml to rss.xml
- svg icon fixes and many new design tweaks
and much more
- fixed issues with metatags
- added "author" tags for HTML and RSS
- fixed XML minification
- minor changes to CSS, HTML, formatting etc
Improved regex usage, sequence of processing, css, and many other things which
resulted in faster execution. Bumped version for dev
Fixed issue with special posts handling
Initial push for version 0.7.1
- Readme.md+73
- index.php+1003
- mistat-cli-0.7.0.py+641
- mistat-cli.py+565
- mistat-fake-data-gen.sh+45