Compiling Old Forum Threads
- jamesrbrown
- Posts: 564
- Joined: Thu May 09, 2013 5:15 am
- Location: Gilbert, AZ, USA
- Contact:
Compiling Old Forum Threads
Did someone say they were going to try to archive the old forum threads? That would be great. It would be a shame if they were lost. Those early discussions and ideas were valuable and revealing.
When will the old forum go away?
When will the old forum go away?
Please visit my blog, Advancement Points: The One Ring Files, for my TOR Resources
Re: Compiling Old Forum Threads
James,
Jon announced that C7 would retain the Forums & at some point attempt to switch them over or save them in some fashion. I believe he said that he might end up doing it (yikes, what a job that will be ).
Jon announced that C7 would retain the Forums & at some point attempt to switch them over or save them in some fashion. I believe he said that he might end up doing it (yikes, what a job that will be ).
Re: Compiling Old Forum Threads
Yes, it will be a complex project... approxiamately 1550 html pages to retain a complete copy of the TOR forums alone.
It's possible that Jon had intended some kind of "cherry picking" effort to excise the best material posted but this becomes subjective and requires reading all 17614 messages posted. Not to mention it might miss the synergy of multiple related topics.
It's possible that Jon had intended some kind of "cherry picking" effort to excise the best material posted but this becomes subjective and requires reading all 17614 messages posted. Not to mention it might miss the synergy of multiple related topics.
Re: Compiling Old Forum Threads
Wow, I hadn't realized what a project it will be. What a hassle for Jon & his crew!
James is right, though: we have so many valuable posts on the previous Forums that it will be a shame to lose them.
James is right, though: we have so many valuable posts on the previous Forums that it will be a shame to lose them.
Re: Compiling Old Forum Threads
Since ClickDev is still experiencing issues, I've already begun to download the TOR Forum topics. More paranoia than anything else; but whatever is happening to ClickDev cannot continue indefinitely. I don't want TOR to suffer in the unlikely event a catastrophic failure occurs. Unlikely, but...
So far the effort is time consuming, frustrating (the 'Server busy, try again' message) and occasionally annoying (when naming conflicts occur) - but has not otherwise been too horrid. I believe TOR was the most active forum and represents the largest potential data loss. So once that is completed other archiving efforts should be simpler.
So far the effort is time consuming, frustrating (the 'Server busy, try again' message) and occasionally annoying (when naming conflicts occur) - but has not otherwise been too horrid. I believe TOR was the most active forum and represents the largest potential data loss. So once that is completed other archiving efforts should be simpler.
Re: Compiling Old Forum Threads
This is wonderful that you're doing this for all of us - thank you!
- jamesrbrown
- Posts: 564
- Joined: Thu May 09, 2013 5:15 am
- Location: Gilbert, AZ, USA
- Contact:
Re: Compiling Old Forum Threads
Awesome Garn! Thanks so much. Curious, what do you mean by 'downloading' the topics and how will you make them available to read afterwards?
Please visit my blog, Advancement Points: The One Ring Files, for my TOR Resources
Re: Compiling Old Forum Threads
Mim & James,
Sorry, I was not clear. I'm using the standard browser "Save Page As..." feature to save a near perfect copy of each topic page (some topics are multiple pages in length). The C7 banner at the top of the page is lost and each message's poster column is just long enough for their name. Resulting in a slightly ragged table appearance as it's no longer a single sized column down the page. Other than that everything else (images, signatures, links, etc) are all retained "as is", assuming they're still available today (so I'm not re-inserting anyone's shared images from lapsed file-sharing accounts even if I personally have them).
Each topic is saved as an HTML page with the topic's title as the filename. Without the sub-titles, which might make the filenames too long (Windows has a 250 char limit), occasionally a duplicate filename occurs. I've added " - DD MMM YYYY" from the 1st message posted to differentiate the topics in this case. For topics with multiple pages, I add " -- pg##" with mandatory leading zeroes (otherwise Windows sorts numbers in the order: 1, 10-19, 100-199, 2, 20-29, 200-299, 3, 30-39, etc). So far, I haven't needed to add a date and page number to the same topic.
Thereafter... I haven't decided what to do with the data. I wasn't sure what other efforts were underway and how complete they might be, so I've assumed that this might be for personal use. But I have no problem making this available to the community. There is no point to multiple people performing the same work.
I thought of just archiving the whole thing as individual HTML pages - exactly as I'm saving them. But your question got me to thinking that would be one of the least efficient means (for searching) of handling the data. Conversion to PDF or EPUB would definitely be a better option, but I'm not sure exactly what this would entail as I've never tried to produce either format from saved web pages, let alone a project of this size, previously. So I cannot be absolutely certain of the feasibility of that portion of the project.
So at the moment, at the absolute least, I should be able to make an archive of the individual HTML pages and post it to a file sharing website of some kind. Presently I have approximately the most recent third of the TOR topics saved to disk. I make no comments with regards to archiving the rest of the C7 Forums - just so that I don't take on more than I can reasonably handle.
Sorry, I was not clear. I'm using the standard browser "Save Page As..." feature to save a near perfect copy of each topic page (some topics are multiple pages in length). The C7 banner at the top of the page is lost and each message's poster column is just long enough for their name. Resulting in a slightly ragged table appearance as it's no longer a single sized column down the page. Other than that everything else (images, signatures, links, etc) are all retained "as is", assuming they're still available today (so I'm not re-inserting anyone's shared images from lapsed file-sharing accounts even if I personally have them).
Each topic is saved as an HTML page with the topic's title as the filename. Without the sub-titles, which might make the filenames too long (Windows has a 250 char limit), occasionally a duplicate filename occurs. I've added " - DD MMM YYYY" from the 1st message posted to differentiate the topics in this case. For topics with multiple pages, I add " -- pg##" with mandatory leading zeroes (otherwise Windows sorts numbers in the order: 1, 10-19, 100-199, 2, 20-29, 200-299, 3, 30-39, etc). So far, I haven't needed to add a date and page number to the same topic.
Thereafter... I haven't decided what to do with the data. I wasn't sure what other efforts were underway and how complete they might be, so I've assumed that this might be for personal use. But I have no problem making this available to the community. There is no point to multiple people performing the same work.
I thought of just archiving the whole thing as individual HTML pages - exactly as I'm saving them. But your question got me to thinking that would be one of the least efficient means (for searching) of handling the data. Conversion to PDF or EPUB would definitely be a better option, but I'm not sure exactly what this would entail as I've never tried to produce either format from saved web pages, let alone a project of this size, previously. So I cannot be absolutely certain of the feasibility of that portion of the project.
So at the moment, at the absolute least, I should be able to make an archive of the individual HTML pages and post it to a file sharing website of some kind. Presently I have approximately the most recent third of the TOR topics saved to disk. I make no comments with regards to archiving the rest of the C7 Forums - just so that I don't take on more than I can reasonably handle.
- Jon Hodgson
- Posts: 1377
- Joined: Wed Apr 24, 2013 11:53 am
- Location: Scotland
Re: Compiling Old Forum Threads
We're still working on making a useable archive of the old forums, but of course I'm sure you can understand we have other priorities which tend to come first. For now they remain accessible, in theory at least, but yes, given the generally frail nature of that service we can't rely on them lasting forever.
There is reportedly an option to pay for a database download of the old forums - ClicDev in their wisdom moved access to such data behind a paywall. When we get a chance we'll look further into that option.
There is reportedly an option to pay for a database download of the old forums - ClicDev in their wisdom moved access to such data behind a paywall. When we get a chance we'll look further into that option.
Re: Compiling Old Forum Threads
Jon, Garn!
Thank you for working on these for us.
Your explanations only emphasize what a massive project this is, though it's worthwhile because of all the great game plots, addendum on rules, & such, that might otherwise fall between the cracks.
Thank you for working on these for us.
Your explanations only emphasize what a massive project this is, though it's worthwhile because of all the great game plots, addendum on rules, & such, that might otherwise fall between the cracks.
Who is online
Users browsing this forum: No registered users and 5 guests