IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Reply to this topicStart new topic
> The Internet Shadowrun Conservation Project (ISCP), Fighting link rot the crowdsourcing way
ThatPaolo
post Aug 29 2009, 07:16 PM
Post #1


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



So, I started putting back the Archive online again and I noticed that a lot of the 2000+ entries there were leading the unwary visitor to the dreary 404 place.

Not good. A lot of Shadowrun knowldege has been lost in the years, with people moving out out colleges, hosting companies shutting down, etc...

So I went and clicked on every single link that was there and kept track of the ones that were still somehow active. It turns out that we still have 150+ live links that - addd to the articles that I was already hosting directly in the Archive - turn out to be a total of 397 valid entries.

150 links pointing to sites that may or may not be around tomorrow. We need to do something.

My idea would be to split these 150 links between a number of Dumpshockers and do the following things:

1. Verify that the link is not a duplicate (I can do this at the database level, so let's skip it for now)
2. Is the site still active?
2.a if not, make a copy of the content
2.b if it is still actively mantained, contact the owner and see if we can create a mirror
3. Notify me with the entry ID, and provide me the content for inclusion in the new Archive

If the owner wants to mantain control over the content, that's fantastic, but this should definitely cut down on dead links and improve the overall usefulness.

So, Dumpshock, what do you think? Are you game? (IMG:style_emoticons/default/smile.gif)
Go to the top of the page
 
+Quote Post
Synner667
post Aug 29 2009, 07:21 PM
Post #2


Moving Target
**

Group: Members
Posts: 942
Joined: 16-September 05
From: London
Member No.: 7,753



good plan

brave of you take it on.

how you plan for people to copy the content ??
how do you plan to mirror the content ??

i might take on some, to do my bit.
Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 29 2009, 07:22 PM
Post #3


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



Alright, I've removed duplicates, so step 1 is done (IMG:style_emoticons/default/smile.gif)
Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 29 2009, 07:24 PM
Post #4


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



QUOTE (Synner667 @ Aug 29 2009, 12:21 PM) *
good plan

brave of you take it on.

how you plan for people to copy the content ??
how do you plan to mirror the content ??

i might take on some, to do my bit.

I don't have a very specific plan for this, so I'm open to suggestions.

Eventually I would like to store all this either in a database or on a set of static pages, maybe a Wordpress site, so people can comment and enrich the contents.
Go to the top of the page
 
+Quote Post
Synner667
post Aug 29 2009, 07:36 PM
Post #5


Moving Target
**

Group: Members
Posts: 942
Joined: 16-September 05
From: London
Member No.: 7,753



hmmm

well, if you can contact the owners, i guess you could just get them d/l the files and rehost them yourself - each in it's own subdomain, maybe

if they know about that sorta stuff

if not, i'm not sure how you can get the files to get the content

plus, if they are dynamic sites any database related stuff will need to be done too...
...tho i guess they can all live on the same database, with different filenames

Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 29 2009, 07:40 PM
Post #6


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



QUOTE (Synner667 @ Aug 29 2009, 12:36 PM) *
hmmm

well, if you can contact the owners, i guess you could just get them d/l the files and rehost them yourself - each in it's own subdomain, maybe

if they know about that sorta stuff

if not, i'm not sure how you can get the files to get the content

plus, if they are dynamic sites any database related stuff will need to be done too...
...tho i guess they can all live on the same database, with different filenames

One option is to just View Source and copy the content to a local html, one file per page. And then include that html in the database (with the proper author, date, title and description, of course)
Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 29 2009, 07:41 PM
Post #7


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



In other news, I removed all the dead links from the archive now. Let me know if you find something bad (IMG:style_emoticons/default/smile.gif)
Go to the top of the page
 
+Quote Post
Synner667
post Aug 29 2009, 07:47 PM
Post #8


Moving Target
**

Group: Members
Posts: 942
Joined: 16-September 05
From: London
Member No.: 7,753



erm, i've never heard of "The Internet Shadowrun Conservation Project (ISCP)", so care to provide a link ??

copying the source misses out on loads of stuff - php tagged stuff won't even show up there, from what i know about coding [which isn't much]

realistically, i think getting owners to send you the files might be best...
...but maybe one of the more learned, techie people on the forums can offer solutions/advice/help
Go to the top of the page
 
+Quote Post
Synner667
post Aug 29 2009, 07:47 PM
Post #9


Moving Target
**

Group: Members
Posts: 942
Joined: 16-September 05
From: London
Member No.: 7,753



erm, i've never heard of "The Internet Shadowrun Conservation Project (ISCP)", so care to provide a link ??

copying the source misses out on loads of stuff - php tagged stuff won't even show up there, from what i know about coding [which isn't much]

realistically, i think getting owners to send you the files might be best...
...but maybe one of the more learned, techie people on the forums can offer solutions/advice/help
Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 29 2009, 07:52 PM
Post #10


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



QUOTE (Synner667 @ Aug 29 2009, 12:47 PM) *
erm, i've never heard of "The Internet Shadowrun Conservation Project (ISCP)", so care to provide a link ??

You are part of it now! (IMG:style_emoticons/default/smile.gif)
QUOTE
copying the source misses out on loads of stuff - php tagged stuff won't even show up there, from what i know about coding [which isn't much]

realistically, i think getting owners to send you the files might be best...
...but maybe one of the more learned, techie people on the forums can offer solutions/advice/help

It's doable, it's just a lot of work. But once we have in a consistent format, then we can use it in so so many ways...
Go to the top of the page
 
+Quote Post
Krypter
post Aug 29 2009, 09:48 PM
Post #11


Target
*

Group: Members
Posts: 91
Joined: 26-February 02
From: Toronto, Canada
Member No.: 1,495



I could help a bit. IME the best format is plaintext; it can be coded or pasted into anything.
Go to the top of the page
 
+Quote Post
Method
post Aug 30 2009, 04:00 AM
Post #12


Street Doc
******

Group: Admin
Posts: 3,485
Joined: 2-March 04
From: Neverwhere
Member No.: 6,114



Wow. The Archive back with the living? That would be great.

Of course, a lot of that content isn't going to be useful any more, what with the new edition and whatnot. I know a lot of my old contributions (bioware, geneware, etc) have been superseded by published material in recent years.
Go to the top of the page
 
+Quote Post
Link
post Aug 30 2009, 12:31 PM
Post #13


Moving Target
**

Group: Members
Posts: 492
Joined: 27-August 02
From: Queensland
Member No.: 3,180



QUOTE (ThatPaolo @ Aug 29 2009, 08:41 PM) *
In other news, I removed all the dead links from the archive now. Let me know if you find something bad (IMG:style_emoticons/default/smile.gif)

I found something bad, all the nostalgic article headings had followed their linked pages and disappeared (IMG:style_emoticons/default/wink.gif) . For posterity's sake (not to mention the inspirational value), how about leaving the article headings on the archive with a different colour or italics to indicate the dead link. Perhaps some old poster will see their article and resubmit it.

As for your project, I'll help once you've decided the process.

QUOTE (Method)
Of course, a lot of that content isn't going to be useful any more, what with the new edition and whatnot. I know a lot of my old contributions (bioware, geneware, etc) have been superseded by published material in recent years.

Not all have gone fourth...
Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 30 2009, 05:41 PM
Post #14


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



QUOTE (Link @ Aug 30 2009, 04:31 AM) *
I found something bad, all the nostalgic article headings had followed their linked pages and disappeared (IMG:style_emoticons/default/wink.gif) . For posterity's sake (not to mention the inspirational value), how about leaving the article headings on the archive with a different colour or italics to indicate the dead link. Perhaps some old poster will see their article and resubmit it.

That makes a lot of sense. Let me see what I can do...
Go to the top of the page
 
+Quote Post
Bull
post Aug 30 2009, 05:51 PM
Post #15


Grumpy Old Ork Decker
*******

Group: Admin
Posts: 3,702
Joined: 26-February 02
From: Orwell, Ohio
Member No.: 50



QUOTE (Link @ Aug 30 2009, 08:31 AM) *
Not all have gone fourth...


Yeah, but there was a lot of 2nd ed material, and I think even a few 1st ed pieces kicking around (IMG:style_emoticons/default/smile.gif)
Go to the top of the page
 
+Quote Post
nezumi
post Aug 30 2009, 06:44 PM
Post #16


Incertum est quo loco te mors expectet;
*********

Group: Dumpshocked
Posts: 6,356
Joined: 24-October 03
From: DeeCee, U.S.
Member No.: 5,760



Yes, excellent! Needed to be done years ago. THank you thank you !!
Go to the top of the page
 
+Quote Post
MusicMan
post Aug 30 2009, 07:42 PM
Post #17


Moving Target
**

Group: Members
Posts: 206
Joined: 25-August 09
Member No.: 17,548



We could archive non-canon material in a tiddlywiki and host it on Tiddlyspot.com

I'm a hyperobsessive/hyperorganisational person with a fair amount of time currently on my hands and wouldn't mind helping out with archiving stuff.
Go to the top of the page
 
+Quote Post
Method
post Aug 30 2009, 09:32 PM
Post #18


Street Doc
******

Group: Admin
Posts: 3,485
Joined: 2-March 04
From: Neverwhere
Member No.: 6,114



QUOTE (Bull @ Aug 30 2009, 09:51 AM) *
Yeah, but there was a lot of 2nd ed material, and I think even a few 1st ed pieces kicking around (IMG:style_emoticons/default/smile.gif)

Yeah that and a lot of stuff was canonized later in SR3 (like the aforementioned geneware in SOTA63)
Go to the top of the page
 
+Quote Post
Xerxos
post Aug 30 2009, 09:32 PM
Post #19


Target
*

Group: Members
Posts: 34
Joined: 17-May 08
From: Germany
Member No.: 15,983



You could also check the Internet Archive if those old sites are still there. The internet doesn't forget that fast.

For mirroring existing but dead sites, there are programs like HTTP Weazel that mirror/download whole sites.
Go to the top of the page
 
+Quote Post
ThatPaolo
post Aug 31 2009, 02:16 AM
Post #20


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



Very good guys, that's the idea (IMG:style_emoticons/default/smile.gif)

I can provide a list of all the active and dead links soon.
Go to the top of the page
 
+Quote Post
coax
post Aug 31 2009, 10:24 AM
Post #21


Target
*

Group: Members
Posts: 2
Joined: 11-May 06
Member No.: 8,548



WHere is the archive now? Even with dead links id like to see it.
Go to the top of the page
 
+Quote Post
ThatPaolo
post Sep 1 2009, 08:20 PM
Post #22


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



http://marcucci.homeserver.com/archive/
Go to the top of the page
 
+Quote Post
ThatPaolo
post Sep 2 2009, 10:14 PM
Post #23


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



A very work in progress of the new archive can be found at:

http://marcucci.homeserver.com/newarchive/

Lots of things are missing and the layout is not final by any stretch of the imagination, but... (IMG:style_emoticons/default/smile.gif)
Go to the top of the page
 
+Quote Post
Krypter
post Sep 3 2009, 01:52 AM
Post #24


Target
*

Group: Members
Posts: 91
Joined: 26-February 02
From: Toronto, Canada
Member No.: 1,495



I'm not sure that structuring the archive into a blog format is the best way to go. All the data is old, so it doesn't really matter whether it was posted in 1997 or 1998. It also makes for a rather bland presentation. Wouldn't a more attractive subject-matter design be easier to navigate?
Go to the top of the page
 
+Quote Post
ThatPaolo
post Sep 4 2009, 12:29 AM
Post #25


the Coke in the Machine
**

Group: Former Moderators
Posts: 124
Joined: 26-February 02
From: Cascadia
Member No.: 46



QUOTE (Krypter @ Sep 2 2009, 05:52 PM) *
I'm not sure that structuring the archive into a blog format is the best way to go. All the data is old, so it doesn't really matter whether it was posted in 1997 or 1998. It also makes for a rather bland presentation. Wouldn't a more attractive subject-matter design be easier to navigate?

What do you propose?
Go to the top of the page
 
+Quote Post

3 Pages V   1 2 3 >
Reply to this topicStart new topic

 



RSS Lo-Fi Version Time is now: 28th August 2014 - 11:17 PM

Topps, Inc has sole ownership of the names, logo, artwork, marks, photographs, sounds, audio, video and/or any proprietary material used in connection with the game Shadowrun. Topps, Inc has granted permission to the Dumpshock Forums to use such names, logos, artwork, marks and/or any proprietary materials for promotional and informational purposes on its website but does not endorse, and is not affiliated with the Dumpshock Forums in any official capacity whatsoever.