STICKY How-to Index
- Thread starter hear
- Start date
this is awesome!
I did it because I figured if I found it useful then most likely somebody else did too. Which isn't to say that I didn't receive a suitable reward for my efforts.
But it honestly didn't take all that much effort to get it where it is right now. I'm working on tweaking a little to make the threads have the proper capitalization etc. And also still awaiting the masses to help categorize the handful that I couldn't immediately classify.
Glad somebody finds it useful besides me.
Thanks for this! Really highlights the amount of great info on this site!
Coachgeo
TJ Enthusiast
there is several mentioned in threads that here in the "How To" is the thread on increasing fuel tank capacity. Am probably blind and missed it.... does not seem to be listed in the index. least not under Fuel system. Did a generic search for Tank in the whole "How To" forum as well. Does not come up with it that either.
Thanks in advance.
Thanks in advance.
I think that's actually just a regular thread in the main forum. With some effort it could probably find its way here.
Which reminds me, I need to go back and actually finish this project, including the part where it identifies new how to's.
Coachgeo
TJ Enthusiast
IMHO put index in Google sheets or similar free sheet or database .. protect it with you only as the editor... and just provide a link to it. Easy for you to update.. easy for folk to search... and wont tie up this server. just a thought
nah, it's got to be programmatic/automated and searchable from within the forum. If you have to go off-site to search, it may as well not even exist.
I have all the data, it's just not laid out exactly how I want it for the next iteration. Really I'm just being stubborn, it's not that much work. I already have the thread scraper & the page generator done. What I need is to have the scraper diff against what I already have to generate the bbcode for the delta of new topics. The issue becomes how to inject new topics into the index. Problem #1 is classification. It would be fun to train an AI to take a stab at it, but I would still want to audit it since the training set is so small it will undoubtedly get a lot wrong. Problem #2 is notification. What's the best way to notify the right people that there are new threads that need classification? And problem #3 is doing the actual update to the index pages. I don't know if I can (or even want to bother) automate editing the existing post with new content.
So yeah. For the time being, just re-running the script to generate bbcode snippets for new topics is probably what I'm going to do, and I'll manually stick them at the end of the index posts and classify them at my leisure. I'm open to how to do all this better.
I have all the data, it's just not laid out exactly how I want it for the next iteration. Really I'm just being stubborn, it's not that much work. I already have the thread scraper & the page generator done. What I need is to have the scraper diff against what I already have to generate the bbcode for the delta of new topics. The issue becomes how to inject new topics into the index. Problem #1 is classification. It would be fun to train an AI to take a stab at it, but I would still want to audit it since the training set is so small it will undoubtedly get a lot wrong. Problem #2 is notification. What's the best way to notify the right people that there are new threads that need classification? And problem #3 is doing the actual update to the index pages. I don't know if I can (or even want to bother) automate editing the existing post with new content.
So yeah. For the time being, just re-running the script to generate bbcode snippets for new topics is probably what I'm going to do, and I'll manually stick them at the end of the index posts and classify them at my leisure. I'm open to how to do all this better.
Jeepster09
Member
Hello friends. I've spent the last few days massively overhauling how the index gets built, most importantly is the ability to identify new threads & update the index. But as a fun side quest I've added an AI element to do the classification. One side effect is that I temporarily (?) had to collapse the "Offroad odds & ends" section, and it dumped more than I expected into the regular odds & ends section. What you're seeing is the result of my first pass of all the threads through OpenAI gpt-4o-mini, which might not be the best choice but it was great on the test data.
I'll share the repo once it gets more refined, but generally speaking it looks like this for now:
* python script which calls the xenforo api to get all the thread metadata (first post, specifically)
* I track the latest post date in mongodb so subsequent runs only pull new threads, and the whole thing stays idempotent
* I publish that to a Redpanda (Kafka, for the uninitiated) topic
* There is a Redpanda Connect pipeline that consumes that topic and looks it up against another mongodb cache, and if not found there calls OpenAI to classify the thread based on the categories you see in the index thread.
* Once classified, those are upserted into yet another mongodb collection which is the system of record for the classified threads
* Another python script takes that collection, sorts it by category, and generates the bbcode markup for the index thread
And then I manually edit the index posts. That can be automated too, but I'm not there yet.
Currently this all runs out of a set of docker containers, but long term I plan to move it to AWS. The thread poll stuff can be a Lambda, and I'll swap out Mongodb for DynamoDB. The Redpanda stuff.....let's just say I can use as much Redpanda as I want w/o any concern. OpenAI is on my dime, but I have literally spent $0.06 between testing and the full classification I just ran. Unless I do another full re-class, the only calls to OpenAI will be when a new thread is created. You'll DDOS the forum before you spend my credits. 
So that's V2.0. Besides the pivot to cloud, I will make some further tech stack refinements to utilize AI agents & MCP to execute this stuff.
I'll share the repo once it gets more refined, but generally speaking it looks like this for now:
* python script which calls the xenforo api to get all the thread metadata (first post, specifically)
* I track the latest post date in mongodb so subsequent runs only pull new threads, and the whole thing stays idempotent
* I publish that to a Redpanda (Kafka, for the uninitiated) topic
* There is a Redpanda Connect pipeline that consumes that topic and looks it up against another mongodb cache, and if not found there calls OpenAI to classify the thread based on the categories you see in the index thread.
* Once classified, those are upserted into yet another mongodb collection which is the system of record for the classified threads
* Another python script takes that collection, sorts it by category, and generates the bbcode markup for the index thread
And then I manually edit the index posts. That can be automated too, but I'm not there yet.
Currently this all runs out of a set of docker containers, but long term I plan to move it to AWS. The thread poll stuff can be a Lambda, and I'll swap out Mongodb for DynamoDB. The Redpanda stuff.....let's just say I can use as much Redpanda as I want w/o any concern.
OpenAI is on my dime, but I have literally spent $0.06 between testing and the full classification I just ran. Unless I do another full re-class, the only calls to OpenAI will be when a new thread is created. You'll DDOS the forum before you spend my credits. So that's V2.0. Besides the pivot to cloud, I will make some further tech stack refinements to utilize AI agents & MCP to execute this stuff.
And since threads without pictures are no good....
This is the raw topic, the output of the xenforo api call.
This is the relevant AI section of the Redpanda Connect AI pipeline:
And then the logging output of that pipeline:
And then the mongodb collection with the classified threads:
This is the raw topic, the output of the xenforo api call.
This is the relevant AI section of the Redpanda Connect AI pipeline:
YAML:
65 processors:
66 - log:
67 level: INFO
68 message: 'OPENAI CALL (cache miss) thread_id=${! meta("thread_id") } title="${! meta("title") }"'
69
70 # tally 1 for every cache miss
71 - metric:
72 type: counter
73 name: tjforum_openai_calls_total
74 labels:
75 model: gpt-4o-mini
76 pipeline: tj_classification
77 value: 1
78
79
80 - mapping: |
81 let cats = [
82 "Axles/Differential","Body/Doors/Exterior","Brakes","Cooling","Electrical",
83 "Engine / Sensors","Frame","Fuel System","Hard top/Soft top","HVAC","Interior",
84 "Lifts/Tires","Lighting","PCM Related","Seats","Sound System","Steering/Suspension",
85 "Transfer Case","Transmission","Off-Road / Trail Mods","Odds & Ends"
86 ]
87 root = {
88 "title": meta("title"),
89 "message": meta("message"),
90 "allowed_categories": $cats
91 }
92
93 - openai_chat_completion:
94 api_key: ${OPENAI_API_KEY}
95 model: gpt-4o-mini
96 system_prompt: |
97 You classify Jeep Wrangler TJ how-to threads into exactly ONE primary category.
98 You MUST choose one from allowed_categories. "Odds & Ends" is acceptable.
99 prompt: |
100 Title: ${! json("title") }
101 Body: ${! json("message") }
102 Allowed categories: ${! json("allowed_categories").join(", ") }
103 Return JSON only.
104 response_format: json_schema
105 json_schema:
106 name: tj_primary_category
107 schema: |
108 {
109 "type":"object",
110 "additionalProperties":false,
111 "properties":{
112 "primary_category":{"type":"string"},
113 "confidence":{"type":"number","minimum":0,"maximum":1},
114 "rationale":{"type":"string"}
115 },
116 "required":["primary_category","confidence","rationale"]
117 }
118And then the logging output of that pipeline:
Code:
rpk connect run cache_classification.yaml
INFO[2025-12-16T01:40:31-06:00] Running main config from specified file @service=redpanda-connect benthos_version=4.69.0 path=cache_classification.yaml
INFO[2025-12-16T01:40:31-06:00] Listening for HTTP requests at: http://0.0.0.0:4195 @service=redpanda-connect
INFO[2025-12-16T01:40:31-06:00] Successfully loaded Redpanda license @service=redpanda-connect expires_at="2028-01-13T12:51:19-06:00" license_org=cnelson license_type=enterprise
INFO[2025-12-16T01:40:31-06:00] Launching a Redpanda Connect instance, use CTRL+C to close @service=redpanda-connect
INFO[2025-12-16T01:40:31-06:00] Output type mongodb is now active @service=redpanda-connect label="" path=root.output.broker.outputs.1
INFO[2025-12-16T01:40:31-06:00] Output type kafka_franz is now active @service=redpanda-connect label="" path=root.output.broker.outputs.0
INFO[2025-12-16T01:40:31-06:00] Input type kafka_franz is now active @service=redpanda-connect label="" path=root.input
INFO[2025-12-16T01:40:36-06:00] OPENAI CALL (cache miss) thread_id=471 title="How to replace the heater core on a Jeep Wrangler TJ" @service=redpanda-connect custom_source=true label="" path=root.pipeline.processors.2.cached.processors.0
INFO[2025-12-16T01:40:36-06:00] OPENAI CALL (cache miss) thread_id=662 title="How to clean your IAC (Idle Air Controller) valve" @service=redpanda-connect custom_source=true label="" path=root.pipeline.processors.2.cached.processors.0
INFO[2025-12-16T01:40:36-06:00] OPENAI CALL (cache miss) thread_id=82 title="Diagnosing and Fixing Death Wobble" @service=redpanda-connect custom_source=true label="" path=root.pipeline.processors.2.cached.processors.0
INFO[2025-12-16T01:40:36-06:00] OPENAI CALL (cache miss) thread_id=85 title="How to align your Jeep Wrangler TJ" @service=redpanda-connect custom_source=true label="" path=root.pipeline.processors.2.cached.processors.0And then the mongodb collection with the classified threads:
I don't speak nerd. Just looking at that makes me dizzy. It's like I can read the words, but I have no idea what's going on. I'm glad you understand it.
WHAT DID YOU CALL ME ?Thems fightin words.
Microprocessor. I'm pretty sure in Indian names that means "Little Wing", or "one with small "wing"".
LOL, since I don't have the audience I thought I did, this is mostly for my own benefit later on. I actually learned a little by re-reading what I wrote when I did v1.0
I changed the prompt and switched to gpt-4.1-mini and I got better results. Would love for some people to peruse the classifications and identify anything they think is wrong. I would pay particular attention to the Offroad and Odds & Ends categories, as those are the hardest to nail. If you're into AI stuff, here's my prompt. If there are any keywords you think should/not be in the offroad category guidance, speak up.
I changed the prompt and switched to gpt-4.1-mini and I got better results. Would love for some people to peruse the classifications and identify anything they think is wrong. I would pay particular attention to the Offroad and Odds & Ends categories, as those are the hardest to nail. If you're into AI stuff, here's my prompt. If there are any keywords you think should/not be in the offroad category guidance, speak up.
Code:
You classify Jeep Wrangler TJ how-to threads into exactly ONE primary category.
You MUST choose exactly one from allowed_categories.
IMPORTANT: "Odds & Ends" is a last-resort category. Only choose it if none of the other categories reasonably fit.
If the thread is about a Jeep TJ modification, repair, maintenance task, troubleshooting, parts install, or upgrade,
you should almost always choose a more specific category than "Odds & Ends". Do not invent new categories.
Category guidance (apply if the category name exists in allowed_categories):
- "Off-Road / Trail Mods":
Use for modifications intended to improve off-road or trail capability/durability, including: lift,
bump stops, control arms, skid plates/armor, rock sliders,
bumpers/winch mounts, recovery points, winches, , lockers,
tummy tuck/high-clearance, articulation improvements, gear/tools storage.
Tie-breakers:
1) Prefer the category that matches the main task a reader would follow the how-to for.
2) If multiple categories could fit, choose the most specific one.
3) Do NOT choose "Odds & Ends" due to vagueness; pick the closest applicable category.Similar threads
