Welcome, Guest!

Here are some links you may find helpful

AssemblerGames Backups

Yakumo

Well-known member
Registered
May 31, 2019
400
332
63
AGName
Yakumo
AG Join Date
From the beginning (whenever that was)
Haha yeah, I remember there not being much. Mostly drama threads/really old posts. A user has copied all the content so we have it just incase :D
Good to know it was possible to access the data just in case there was something worthwhile.
 
  • Love
Reactions: 1

Alpha

Member
Registered
Oct 15, 2018
17
35
13
As luck would have it, I already wrote a crawler last year for AG that will convert the public posts/threads/forums into a MySQL database. I'm running it now and will see about posting the dump once it finishes.

3IckoNv.png
 
1

1

Guest
Original poster
As luck would have it, I already wrote a crawler last year for AG that will convert the public posts/threads/forums into a MySQL database. I'm running it now and will see about posting the dump once it finishes.

3IckoNv.png

Having it hosted in a MySQL database would be really handy, I'm hoping that vBulletin3 database I sent is enough :)

EDIT: Running Alpha's script too so we'll have three current post backups.
 
Last edited by a moderator:
  • Like
Reactions: sheltontristania

Nemesis

Well-known member
Registered
May 30, 2019
58
125
33
AGName
Nemesis
AG Join Date
Mar 22, 2007
Any thoughts about loading the content back in to a new forum? I'd thought about the scraping idea, but hadn't bothered. If you've already done it, what about going a step further? If we have a database of all the threads, posts, account names, avatars, etc, it should be possible to write a script that populates a new forum database with that content. You could create legacy accounts like "AG_Nemesis" to "own" the old posts, and blast the content into a new forum.
 
  • Like
Reactions: 1
1

1

Guest
Original poster
Any thoughts about loading the content back in to a new forum? I'd thought about the scraping idea, but hadn't bothered. If you've already done it, what about going a step further? If we have a database of all the threads, posts, account names, avatars, etc, it should be possible to write a script that populates a new forum database with that content. You could create legacy accounts like "AG_Nemesis" to "own" the old posts, and blast the content into a new forum.

While this would be a good idea much of the older content isn't much value but alot is at the same time. Currently with Alpha's script we're making a database of posts/threads/usernames and the intention is to make a static clone of AG on another domain. This would be controlled via MySQL if there was ever a need to merge here but it wouldn't be able to be logged into. Akira is helping there with that and I'll be doing the html/css side to make it look like AG minus images etc although I may manually add those not sure yet.
 

Chaotic Mind

AG Refugee Retro Brony
Registered
Jun 1, 2019
70
59
18
AGName
sonicdude10
AG Join Date
Jan 17, 2012
I might can help with doing more archiving if needed. I have terabytes of storage space and gigabit internet that will give me full speed for both DL and upload. PM me if that would be useful to archive anything else. Only hidden stuff I can access on AG is the 0th bit.
 
  • Like
Reactions: sheltontristania

Nemesis

Well-known member
Registered
May 30, 2019
58
125
33
AGName
Nemesis
AG Join Date
Mar 22, 2007
While this would be a good idea much of the older content isn't much value but alot is at the same time. Currently with Alpha's script we're making a database of posts/threads/usernames and the intention is to make a static clone of AG on another domain. This would be controlled via MySQL if there was ever a need to merge here but it wouldn't be able to be logged into. Akira is helping there with that and I'll be doing the html/css side to make it look like AG minus images etc although I may manually add those not sure yet.
Be aware that the backup I did in December last year is a perfect clone of the visual content of the site in a 100% standalone fashion (IE, you can load it in your local browser offline and it'll appear identical to the real site). I've just done a full mirror of the site again using the same process, which I'll upload soon. If you just want to make a static site mirror, that clone will function well in that regard. I don't have access to the 0th bit section to incorporate that into the mirror, but if someone is willing to lend me access to their assemblergames account who has access to that section, I could incorporate that too. I'll be publishing instructions and tools to help people backup their PMs in the next little while too. I had hundreds of PMs I wanted to save, so I've rigged up a more limited mirror that'll function to capture just that content fully intact (IE, images, attachments, and so on).
 

Chaotic Mind

AG Refugee Retro Brony
Registered
Jun 1, 2019
70
59
18
AGName
sonicdude10
AG Join Date
Jan 17, 2012
Be aware that the backup I did in December last year is a perfect clone of the visual content of the site in a 100% standalone fashion (IE, you can load it in your local browser offline and it'll appear identical to the real site). I've just done a full mirror of the site again using the same process, which I'll upload soon. If you just want to make a static site mirror, that clone will function well in that regard. I don't have access to the 0th bit section to incorporate that into the mirror, but if someone is willing to lend me access to their assemblergames account who has access to that section, I could incorporate that too. I'll be publishing instructions and tools to help people backup their PMs in the next little while too. I had hundreds of PMs I wanted to save, so I've rigged up a more limited mirror that'll function to capture just that content fully intact (IE, images, attachments, and so on).
Check PM.

EDIT: I can get a domain in 2 days when I have money. ASSEMblerarchives is available in every way if we wanna use that to host the archived site under. Will cost me $12 USD a year for the domain if I use my Google domains to get it.
 
Last edited:
  • Like
Reactions: sheltontristania

Taucias

New member
Jun 2, 2019
4
9
3
That's okay. There wasn't that much worth bothering with in that section either. Funny thing, when we quit being mods we were kicked out of the 0 bit section.

Nothing changed after you guys were locked out, in terms of new content.
 

Chaotic Mind

AG Refugee Retro Brony
Registered
Jun 1, 2019
70
59
18
AGName
sonicdude10
AG Join Date
Jan 17, 2012
Yeah. There has been no activity in there until recently. I was the last to post in the roll call thread. That was several years back. Makes me wonder if I'm one of the last to be given the role.
 

Bad_Ad84

Well-known member
Registered
May 30, 2019
85
202
33
AGName
Bad_Ad84
AG Join Date
26/05/2011
Check PM.

EDIT: I can get a domain in 2 days when I have money. ASSEMblerarchives is available in every way if we wanna use that to host the archived site under. Will cost me $12 USD a year for the domain if I use my Google domains to get it.

We have a suitable domain already, which will host the mirror.
 
  • Like
Reactions: porchmonkey

Chaotic Mind

AG Refugee Retro Brony
Registered
Jun 1, 2019
70
59
18
AGName
sonicdude10
AG Join Date
Jan 17, 2012
OK. That's good. You all here went the whole 10 yards with this.
 
  • Like
Reactions: 1

Seroczynski

Member
Registered
May 31, 2019
21
2
3
AGName
Seroczynski
AG Join Date
Jun 30, 2016
Really nice work on the script Alpha, I'm interested in what the end-result be like here.
 
1

1

Guest
Original poster
Added the May 29th WGET archive to first post! it's 17GB compressed :|
 
  • Like
Reactions: kabojnk
1

1

Guest
Original poster


Progress on the @Alpha crawler rip. About 40k threads, 700k threads left and about 17k members before it finishes. Akira's already working on the front end to display the data in a static version of AG on a domain we've purchased in advanced. Only thing to add to that is a search system which another members working on.
 
  • Like
Reactions: Flaya and FamilyGuy

Nemesis

Well-known member
Registered
May 30, 2019
58
125
33
AGName
Nemesis
AG Join Date
Mar 22, 2007
You should be aware, I'm about to drop my backup of the site, which is more complete and ready for use than the wget mirror you've posted. Backup is done, cleanup is done, archive is compressed, just needs uploading, which I'll get done in the next couple of hours.
 
  • Like
Reactions: Shane McRetro
1

1

Guest
Original poster
You should be aware, I'm about to drop my backup of the site, which is more complete and ready for use than the wget mirror you've posted. Backup is done, cleanup is done, archive is compressed, just needs uploading, which I'll get done in the next couple of hours.

We've got an HTTRACK Mirror running too, as mentioned in first post :) along with a custom built crawler that's auto populating a MySQL database, check the post in news once I'm done for more info thanks, yours wll be compared just to be sure ours is good before sticking on archive.org.
 

Nemesis

Well-known member
Registered
May 30, 2019
58
125
33
AGName
Nemesis
AG Join Date
Mar 22, 2007
Alrighty, just don't underestimate how pedantic I am ;). This is no ordinary build of httrack, and I spent over a month perfecting the ripping process last year. For example, I made sure to rip all the forum index pages in a 5 minute window, then checked back on the site to ensure no topics fell through the cracks by a new thread being added to page 1 for example, before I'd retrieved page 64. I made sure every single page of every topic got scanned, and got scanned only once, and I wrote tools to verify the results. I renamed attachments, ripped all important external content (such as included css and embedded images), and spent hours browsing the backup with no internet connection to make sure it looked identical to the real site. I made sure you could view poll results, use the dynamic "arrows" to build valid page navigation links when browsing long topics/forums, and I even made sure the favicon is properly referenced on every page so that it appears for the complete site, even if you open it locally or don't host it at a root domain level. I had to make lots of fixes to the engine too, in order to make sure topics with unicode characters in the URL like this one got mirrored properly, as well as fix a bunch of other issues. The list of rules is so long I couldn't pass it as command line arguments normally, as it blew the character limit, I had to launch the process through code to get the arguments in. Basically, I obsessed over it and spent far too long making sure it was just right, and I've been through that process again now, with improvements over last time. I'd strongly recommend you try out my rip (with your internet disconnected!) before you consider hosting anything. I think you'll find there's not much else to do than drop the files on a webhost and walk away.
 

Make a donation