Request to update the forums readability system.

[Joshe]

You gotta download Claude Code CLI and then just send it that. It will build it for you. 

[Basman]

15 hours ago, Joshe said:

lol, the solution to your problem. 

Bald Mcgod bro can fix it himself

[glassfire]

it's like reinventing the holy jewish book. We don't need other books

[Lucasxp64]

21 hours ago, Joshe said:

Just have Claude Code do it for you. Here's a prompt that should get you an app to fetch the blog daily and let you search it, all hosted locally:
 

From the scraping, somebody should just generate an RSS feed file with the full posts embedded and with the OCR idea in a free static file hosting service like Cloud flare. So the user can just use his favorite RSS reader. There are probably some good ones that allows for full text search, if not that can be easily be done from the RSS feed file. This way we don't have to waste time maintaining more custom code than necessary, we won't be running multiple of those bots accessing and creating their own databases, and it will work in all platforms and provide the real-time update abstraction from RSS feed readers.

If that's not enough, we can just write our own custom format if RSS happens to have some major limitation like maximum number of posts.

Extracting the posts into their own respective HTML files cleaned and properly tagged for automation before doing any further format transformation should be the first step of this scraping pipeline, that will guarantee that it will preserve all the original features on the post, like the youtube embeds and whatever he might have included there.

The major issue I believe would be the vimeo embeds, I believe leo configured them to not run outside of actualized.org. It has to be checked.

What Claude Code gave there wouldn't handle if it's taking just the text of the posts and tossing away the original HTML.

HTML already has a tag to add the text description of images that is used for accesability, that would be the correct way to extracting HTML and using OCR to embed back into the image the text, and then the later stages of the pipeline uses that.

 

For searching quick right now just use on google: the tag "site:" followed by the address to search there.

site:https://www.actualized.org/insights

Edited 1 hour ago by Lucasxp64

[Joshe]

57 minutes ago, Lucasxp64 said:

From the scraping, somebody should just generate an RSS feed file with the full posts embedded and with the OCR idea in a free static file hosting service like Cloud flare. So the user can just use his favorite RSS reader. There are probably some good ones that allows for full text search, if not that can be easily be done from the RSS feed file. This way we don't have to waste time maintaining more custom code than necessary, we won't be running multiple of those bots accessing and creating their own databases, and it will work in all platforms and provide the real-time update abstraction from RSS feed readers.

The recon didn't find any RSS feed. 

1 hour ago, Lucasxp64 said:

Extracting the posts into their own respective HTML files cleaned and properly tagged for automation before doing any further format transformation should be the first step of this scraping pipeline, that will guarantee that it will preserve all the original features on the post, like the youtube embeds and whatever he might have included there.

Yeah, the prompt intentionally omitted the scraping strategy so it could figure it out on the fly, as opposed to boxing it in early. 

1 hour ago, Lucasxp64 said:

The major issue I believe would be the vimeo embeds, I believe leo configured them to not run outside of actualized.org. It has to be checked.

I haven't tried lately, but pretty sure yt-dlp can download Vimeo vids. Works for my porn anyway, lol. 

1 hour ago, Lucasxp64 said:

What Claude Code gave there wouldn't handle if it's taking just the text of the posts and tossing away the original HTML.

HTML already has a tag to add the text description of images that is used for accesability, that would be the correct way to extracting HTML and using OCR to embed back into the image the text, and then the later stages of the pipeline uses that.

Claude knows this is a GUI app with a backend. It knows it would need to track the markup to display it like like an actual blog post. You can't display videos and images with raw text. I mean, you could, but it wouldn't make sense because you'd need to style the content anyway. I'm pretty sure it would do recon and build a parser based on its findings and then reconstruct the markup based on user requirements. You wouldn't want to keep Leo's markup. It's probably shit, lol.

Good idea about putting the OCR capture in the alt attribute, although not sure if that's beneficial for search. I'm no expert though.