Shadowrun 3 Index, Was it ever attempted? Options

[Sahandrian]

I asked this once in the main SR3R thread, but it was off-topic and easily missed, so I guess I'll repeat here.

There used to be a bit of talk about a fanproject of making a large index to cross-reference all the SR3 rulebooks, and give an index to the ones without.

Does anyone know if this was ever done, or even attempted?

I've seen a number of comments spread through different threads, but never anything come out of it.

[mfb]

whoah, whoah, whoah. easy to find? cross-referencing? YOU'RE KILLING THE SPIRIT OF SHADOWRUN!

[Link]

If this is an effort to start this as a com-pro, I'll help. The index should be for SR 1,2 & 3 though, now that they won't be surprising us with any new books it can be a complete reference.
Is there software for this sort of thing? How would one go about putting it together?

[farfromhomefish]

[Link]

MySQL...YourSQL whatever it takes :D
Ideally one could search online but a printable super index would be good as well. I'd do data entry if it were relatively non tedious ie page by page rather than individual entries, I'm not sure how the software operates.
My thoughts were to scan, OCR or copy indexes (and perhaps contents) from all books and sort. Those with official pdfs could easily submit a copy. I have already gathered half a dozen in thinking how this might be done.

[Sahandrian]

Using PDFs makes it legally questionable - either for scanning pages or for pulling pages out of a published PDF.

I think it comes down to three possibilities. PHP, a wiki, or just using HTML with links between categories.

If there's already a Shadowrun-specific wiki out there, that would probably be the simplest way to do it, especially for group contributions. HTML, on the other hand, would also be very fast and simple, but might not look as good, and would be a bit tougher to edit.

I don't know PHP, but I've been told MySQL is similar to MS Access, which I have used. If so, that would probably function best, but again probably be difficult, and retain the lack of open editing.

[Link]

[Westiex]

A lot of the equipment has already been compiled. Look at Man and Machine for cyberware, bioware and chemicals. Cannon for weapons. Matrix for decking and computer gear ...

[Link]

I was thinking of the ultimate SR gear list as an expression of fair use rather than something we would make for the index.
I just noticed the other day that you sent a PM about 2 years ago regarding MIJI rules - still want 'em? Am I the only one around who never checks PM's :embarassed:

[Sahandrian]

Plus you can't search a scanned PDF.

[farfromhomefish]

[Bastlynn]

The biggest advantage of a wiki would be that there's a relatively low learning curve for the contributers. In any sizable volunteer project you've got to consider that factor or watch your project die a slow agonizing death. On the other hand, given that the material won't be expanding at a terribly huge rate if you can keep interest high to start then you should be able to get the initial input in easily...

Unfortunately a wiki is almost too broad for what is called for in this sort of project - and certainly doesn't have a way to store the background relationship information for easy index - if you wanted to make something simple and searchable for just the information you need in an index - you would need to make a custom application. I could see the following tables in it:

keywords: id, word - id is autoincremented key
books: id, name, abbreviation, edition_number - id is autoincremented key
index: keyword_id, book_id, page_number - all three values are key

You probably would want the end-user application to have a page where you can download a pdf of a master document. And a page to search through the index for specific keywords, or to list words found in specific books. I would suggest having the php page generate the list of indexes out in either html or pdf - searcher's choice (there's enough php classes out there to convert html to pdf quickly, that this feature is almost negligible for a decent programmer). If the only information stored is index and keyword then there's no reason to do any cross-linking.

An OCR of content pages could certainly make for a good start to any indexing project, and it may be worthwhile to write a quick input page to take in these files, but that can be a tricky deal depending on the format of the file. You can also have a page that allows users to enter the data by hand, but that can be tedious, though it's not like tedium isn't a built in part of this sort of thing. Open community editing on the fly is actually the easier of the options than restricted editing (since restricted editing would require someone to write in a login and password system as well).

But all of that assumes you have a programmer and a hosting site that allows you use of php and mysql (geocities freebie accounts need not apply) - so if you don't - you're pretty much stuck with what's out there.

In which case, wiki it is. In that case you may want to approach it less like an index (boring to read topics) and more like an encyclopedia. Each topic page has a *very* brief overlook - and then cites references at the end of the entry by book, edition and page number. A little more indirect, but sans programming something custom - it's what you've got. You won't be able to easily print it out in any sensical format.

As for how complex it would be to modify MediaWiki - at this point I would say: Infinitely complex. You haven't really defined what you *want* the interface to be. Without a target, no programmer could even begin to take a shot.

[Link]

To help those who have proficiency with various applications discussed here to decide what course to take, here is a copy of the index from the Corporate Security Handbook using OCR. The ideal platform would let us enter data in this format as it is not impractical to collect indexes from the few dozen SR books.
Discuss :)

[ Spoiler ]

A
Access control 28-29
biological recognition system 31
maglocks 29-30
pass system 30-31
Active infrared lighting 17
Adept
executive protection 107
physical 61 104
Enhanced Perception power 104
Extended Missile Parry power 104
Air-pressure detectors 27-28
Alarms
electromechanical 26
magical 40-41
Alarm systems 26
area-detection 27-28
perimeter 26-27
proximity 28
Animals 23
handler 120
Ares catalog 66-93
Artificial Sensory Induction System Technology (ASIST) 48
Assailants 58
Assassination 58
Astral containment net 40
Astral patrolling 103
Attack, potential forms of 58-59
B
Bacterial Containment Grid (BCG) 84
Barbed wire 14
Barriers
manmade 14-16 98-99
natural 14 98
spells 36-37 40
Biological recognition systems 31
Blackmail 58
Building defenses 18
containment 20-21
doors 19-20
keys 20
locks 19-20
neutralizing enemy 20-22
walls 19-20
windows 18-19
C
Capacitance sensors 26 28
Cardreader 30 101
Chain-link fencing 15
Chemical compounds 81-83
detectors 33
Closed-circuit simsense (CCSS) 76
history of 48
rules 102
system 31 32
Closed-circuit television (CCTV) 31
Closed-circuit trideo (CCT) 31
Computer virus 10 46-47
Concertina-wire fences 15
Containment measures 20-21
Control systems 31
closed-circuit simsense 32
devices 32-33
Counterintelligence 52-53
Countermeasures 40-41
Cyberware scanner 33
D
Dataline tap 105
Decker combat rules 105
executive protection 61 108
matrix security 49 116
Defenses
animals 23-24
building 18-22
guards 22-23
mechanical 18
perimeter 14-18
Dogs 23
Doors 19-20
Drones 89-93
physical security 18
technical security 32
E
Echo Mirage project 10 46-47
Electrical switches 26
Electrified wire 14-15
Electromechanical alarms 26
Elementals 41
Encryption 105
Enemy neutralization 21-22
Executive protection 54 56
lifestyle modification 59
measures 59-60
personnel 56
adept 61 104 107
decker 61 108
mage 109
magician 61
rigger 61 110
team leader 60-61
rules 104
threat analysis 56-59
Executive-protection specialist (EPS) 61 110 121
F
Fat bacteria (FAB) 14 39-40
netgun 70 103
rules 103-4
strain 1 (FAB-1) 82
ultraviolet (FAB-UV) 39 83
rules 103-4
FAB zones 38-40
Fences 15
Fiber-optic image links 38 102-3
Fiber-Optic Observation Network 75
Fluorescent lights 17
Free-ranging drones 32
G
Gaseous-discharge lighting 17
Geese 23
Glass, for windows 18
Goblinization 9
Guards
magical security 41
physical security 22-23 117
Gun ports 18 21
H
Hardwire access point 105
Hermetic magicians 42
I
Identification cards 31
Incandescent lights 17
Individualized Biometric Safety (IBS) 71
Induced living walls 37
Information, control measures 60
Integrated Control Center (ICC) 74
Intelligence-gathering program 60
Intelligence operations 52-53
Intrusion countermeasures (IC) 10 46 49
Investigations 51-52
K
Keypad 30
breaking 100-101
rules 100-101
Keys 20
Kidnapping 58
Knight, Damian 10
Knight Errant Security Services 10-11
Knockout gas 22
L
Landscaping, defensive 15-16
Laser mazes 21-22
Lifestyle modification 59
Lighting 16-17 72
Living mesh 38
Living nets 38
Living restraints 38
Living walls 14 36 37
Locks 19-20
Lone Star Security, Inc 10
M
Mages
executive protection 109
magical security 41-43 118
Magical alarms 40-41
Magical security 34 36
astral patrolling 103
and executive protection magician 61
measures 60
fat bacteria 103-4
fiber-optic image links 102-3
levels 96
personnel 42
engineer 113
freelance consultant 122
guards 41
mages 41-43 118
physical defense 42
research 42
specialist 112
wards 41-42
rules 102-4
technological developments 36-41
Maglocks 29 73
management of 30
rules 100-101
styles of 30
types of 29-30
Manual switches 17
Matrix, development of 10
Matrix security 10 44
history of 46-48
levels 96
personnel decker 49 116
system design 48-49
Mazes 16 21-22
Mechanical defenses 18
Mechanical switches 26
Microwave motion detectors 27
Moat 15
Monowire 14
Monowire mazes 22
Motion detectors 27-28
rules 99-100
Muscular Signal Transference (MST) 48
N
Nature spirits 41
Netguns 21 70 103
No man's land 16
O
Overall security level (OSL) 98
P
Padlock 20
Passcard 101
Passive alert 64-65
Passive-infrared detectors 27
Pass systems 30-31
Perception Test
wire modifiers 98
Perimeter alarms 26-27 64 65
rules 99-100
Perimeter defenses 14
barriers 14-16
rules 98-99
lighting 16-17
mechanical defenses 18
sound systems 17-18
Personnel
animal handler 120
design engineer 124
executive protection 56 107-10
adept 61 104 107
decker 61 108
mage 109
rigger 110
specialists 61 110 121
team leader 60-61
fast response officer 120
magical security 41-43
engineer 113
freelance consultant 122
guards 41
mages 41-43 118
physical defense 42
research 42
specialist 112
wards 41-42
matrix security decker 49 116
non-player characters 106-25
physical defense 42
physical security 22-23
freelance rigger 123
guard 22-23 117
research 42
screening of 51
security officer 119
technical security
rigger 119
testing of 51
training of 8-9
Personnel
security 50
counterintelligence 52-53
executive 116
intelligence operations 52-53
investigations 51-52
levels of 96 98
personnel
investigator 11I
specialist 114
security awareness programs 53
Photoelectric cells 17
Photoelectric devices 27
Physical print scanner 101
Physical security 13
barrier spells and 37
building defenses 18-22
executive protection measures 59-60
levels 96
perimeter defenses 14-18 98-99
personnel 22-23
freelance rigger 123
guard 22-23 I17
rules 98-99
wire 98-99
Physical threats 58
Pin-tumbler lock 20
Plastic, for windows 19
Polycarbonate glazing 18
Pressure devices 26-27
Pressure mesh rules 100
Pressure pad rules 100
Print scanner 101
Private security firms
history of 7-8
networking by 8
training 8-9
Protocols, adjusting to 105
Proximity alarms 28
Q
Quartz lamps 17
R
Reinforced armor glass 18
Research personnel 42
Residential analysis 57
Retinal print scanner 101
Rigger
combat rules 104-5
Decryption Module 80
executive protection 61 110
freelance 123
matrix security 48
Protocol Emulation Module 79 105
Protocol Emulation Utility 77
technical security 119
Roleplaying 104
S
Safety glass 18
Security
history of 7-10
levels of 96
magic's influence on 9-10
modern services 10-11
Security awareness programs 53 59
Security system
defeat of 104-5
design engineer 124
rules 94-105
Semi-mobile drones 32
Sensors
perimeter 26-28
rules 99-100
Shiawase Decision 9
Shutters 21
Sound systems 17-18
Specialists
animal handler 120
executive protection 61 110 121
fast response officer 120
magical security 112
personnel security 114
security awareness 53
Special Weapons and Tactics (SWAT) 120
Spells
barrier 36-37 40
detection 40
stun 40
thunderclap 40
Strategic location scouting 14
Surveillance systems 31
closed-circuit imaging 31
closed-circuit simsense 32
devices 32-33
T
Taut-wire detectors 26
Team leader 60-61
Technical security 24 26
access control 28-31
alarm systems 26-28
closed-circuit simsense 102
control systems 31-33
device ratings 99 100
executive protection measures 59-60
levels of 96
maglocks 100-101
perimeter alarms 99-100
personnel
rigger 119
rules 99-102
surveillance systems 31-33
vibration detectors 100
voice recognition systems 101-2
Tempered glass 18
Threat analysis 56-59
Timed switches 17
Training, paramilitary 8-9
Transportation, security measures 60
Traps 16
tiger-pit 16
Trip beams 98-99
Trip wires 98
U
Ultrasonic motion detectors 27
Ultrasound emitter/detector 99-100
Unexplained Genetic Expression (UGE) 9
Unfriendly extraction 58
United States v Seretech Corporation 6 9
V
Vibration detectors
devices 28
rules 100
Virally Induced Toxic Allergy Syndrome (VITAS) 9
Voice
mask system 102
recognition systems 101-2
W
Walls 14-15 19
Ward check 64
Wards 37 41-42
Watchers 41
Weapon detectors 32
White-noise generators 22
Wilson, Clay 10
Window foil 26
Windows 18-19
Wire 14-15
rules 98-99
Wired glass 18
Workplace analysis 57

[Bastlynn]

Hm... looks like it's pulling the information in as a tab delineated list - which means it can be translated *quite* well by php's built in functions ...

*sigh* *hangs head and feels guilty* Me and my taking on of projects...

Ok - someone find some hosting... ;) I'm not sure if the folks here at dumpshock would be up for it or not but it's worth asking. I can probably crunch an initial version out within a day or so. I need to wrap up a project for a job first is all.

[Sahandrian]

I originally intended to try and do it myself, if it didn't exist, but at this point it's gone beyond my ability, at least in the programming area. Sometimes I wish I'd stuck with my original CS major (and not just because I'm in school an extra year due to the switch).

I could still try and type up some of the indexes, at least if there are some books I have that those with access to a scanner don't.

And what about the books without indexes? Work one out ourselves?

Edit: Oh. And I'm no use for hosting. I just have Geocities.

This post has been edited by Sahandrian: Jun 13 2007, 01:34 PM

[Link]

When compiling the Corporate Security Handbook index I put a csh entry on each row after the page reference(s) for sorting purposes.
eg. Access control 28-29 csh
As for hosting, my ISP provides 10mb homepage allowance (is this what you're talking about?)

[Chibu]

Hey, as per usual, I should be able to host anything you need. my little webserver gets pretty good speed and I'm not likely to run out of space any time soon. So, if you'd like I'm sure we can get sometime set up.

So for the books that don't have an index, I was thinking two things. First, go through and note whenever any of the words in the other indexes are used =\ yeah annoying, i know. And also add other words that might be useful.

to do this... it would probably be best to convert one of the books to text and then search (ya know, with a search function) through it and note whenever any specific words are used.

Or, if you get a notepad and start typing as you read through the book and on each line just put the keyword you find, a tab, and the page number, and keep then in no sort of order, I'm sure we (myself if you what) could whip up a little script to take that text file and alphabetize it and make an index from it.

Also, I can probably cobble together a DB to store the index, and after that a search script or a complete index would be rather simple to make. all that's needed for the index table is:

id, word, source_book, and page number

And then make the words able to be in there more than once, and then just pull them out all at the same time. But yeah, I'm up for helping if anyone wants.

[Bastlynn]

I suspect chibu you would find in practice that your table design won't be as efficient as you would like: you'll end up repeating words in different books, as well as books for each word, and cross referencing between books and words could be a major CPU drain.

What I was thinking for the books that don't already have an index is unfortunately it would have to be by hand by someone. If someone can create a CSV or tab delineated file out of Excel then they could upload it all in one chunk - it's basically the same thing as the file created by OCR...

How would we be able to retain the page number if the book were converted to text file? Just have the person doing the converting make note of it?... oh. Idea! This would require some familiarity with PDFs and Word though but it would be muuuuch faster if someone did it this way...

Ok - this assumes that whatever PDF you're working with has text or has been scanned with OCR to create a PDF with text, and that you have a full copy of Adobe Acrobat... You *should* then be able to export the file out to Word to create a .doc file with the exact same page numbering. MS Word has the ability to take a set of index words in a separate file and auto create an index. (Look it up in Word's help files to learn how.) So that would allow you to just create an index out of thin air without spending hours on hours reading through it. It would just need a quick once over to be sure it got everything. Then - *that* file can be pulled back into PDF for conversion via OCR or have the created index copied and pasted into Excel to create your tab delineated file for uploading.

Unfortunately if it's a book that currently has no PDF - then the old fashioned way will probably have to do. I can't think of a way to make that easier.

[Link]

There are plenty of books with indices/indexes and for those that don't hopefully some of them are suitable pdf's.
I retrieved the list of SR books from the rpg site for reference. We can work out what we have and put out a list of those indexes someone may be able to supply.

[ Spoiler ]

# First edition
7100 SR (First Edition)
7102 SR1 GM Screen (Silver Angel) adventure
7104 Street Samurai Catalogue
7106 The Grimoire
7107 Virtual Realities
7108 Rigger Black Book
Second edition
7900 SR Second Edition
7902 SR2 GM Screen
7903 Grimoire Second Edition
7904 Virtual Realities 2.0
7905 SR Companion
7906 Rigger 2
Third edition
7000 SR Third Edition
7002 GM’s Screen & Critters Sourcebook 25008
7003 SR Quick Start Rules
7126 Man & Machine 10663 25001
7905 SR Companion (3rd Ed) 10656 25010
7907 Magic in the Shadows 10658
7908 Cannon Companion 10659
7909 Matrix
7910 Rigger 3
10662 Rigger 3 Revised
Fourth edition
26000 SR, Fourth Edition
26001 SR, Fourth Edition Limited Edition
26002 GM's Screen w Contacts and Adventures
26004 Street Magic
26006 Character Dossier (to be released)
26007 Arsenal (to be released)
26009 Augmentation (to be released)
General sourcebooks
First & second edition
7103 Sprawl Sites
7104 Street Samurai Catalog
7104 Street Samurai Catalog (revised for 2nd Ed)
7105 Paranormal Animals of North America
7107 Virtual Realities
7108 Rigger Black Book
7109 Shadowbeat
7110 Shadowtech
7112 Paranormal Animals of Europe
7113 Corporate Shadowfiles
7114 Fields of Fire
7115 Lone Star
7116 Prime Runners
7117 Bug City
7118 Corporate Security Handbook
7119 Cybertechnology
7120 Awakenings
7121 Threats
7122 Portfolio of a Dragon: Dunkelzahn's Secrets
7123 Underworld Sourcebook
7124 Cyberpirates
7125 Corporate Download
7208 The Neo-Anarchists Guide to Real Life
7219 Target: Matrix
Third edition
10650 Year of the Comet
10651 Target: Awakened Lands
10652 Threats 2
10653 Target: Wastelands
10654 Wake of the Comet
10664 State of the Art: 2063
10665 Survival of the Fittest
10666 Dragons of the Sixth World
10667 Sprawl Survival Guide
10673 The SR Character Dossier
25003 Mr. Johnson's Little Black Book
25004 State of the Art: 2064
25006 Loose Alliances
25013 State of the Art: 2063 (Reprint of 10664)
25014 System Failure
Fourth edition
26011 Emergence (to be released)
Location sourcebooks
7201 Seattle Sourcebook
7202 Native American Nations Volume One
7203 London
7204 Germany Sourcebook
7206 Neo-Anarchists Guide to North America
7207 Native American Nations Volume Two
7209 California Free State
7210 Tir Tairngire
7211 Tir na nOg
7212 Denver: The City of Shadows
7213 Aztlan
7214 Target: UCAS
7215 Target: Smugglers Havens
7216 New Seattle 10657 25009
25007 Shadows of Asia
10655 Shadows of North America 25015
25002 Shadows of Europe
25011 Shadows of Latin America (.pdf)
26005 Runner Havens
Adventures
7205 Universal Brotherhood
7301 DNA/DOA
7302 Mercurial
7303 Dreamchipper
7304 Queen Euphoria
7305 Bottled Demon
7306 Harlequin
7307 Dragon Hunt
7308 Total Eclipse
7309 Imago
7310 Elven Fire
7311 Ivy & Chrome
7312 One Stage Before
7313 Dark Angel
7314 A Killing Glare
7315 Celtic Double Cross
7316 Eye Witness
7317 Paradise Lost
7318 Divided Assets
7319 Double Exposure
7320 Harlequins Back
7322 Super Tuesday
7323 Shadows of the Underworld
7324 Predator and Prey
7325 Missions
7326 Mob War
7327 Blood in the Boardroom
7328 Renraku Arcology: Shutdown
7329 First Run
7330 Corporate Punishment
7331 Brainscan
26003 On The Run
Miscellaneous items
7111 Downtown Militarized Zone
7401 Sprawl Maps
7701 High Tech & Low Life - The Art of SR

Just to clarify, is the index document format noted above (eg. Access control 28-29 csh - ad nauseum) suitable? I'd do a few more books if so.

[Bastlynn]

Yes.
So long as it is a regular repeating pattern I can get a php script to recognize it. In this case it's 'keywords' followed by a tab or two, followed by 'page or pages separated by a dash', and the line ends with a standard breakline so... anything that doesn't match that pattern can be ignored. :)

I would not suggest adding the abbreviation for the book t the end of that line - just use what OCR provided. The page that'll ask for the file to be uploaded will ask you what book it came from. That should save you some typing.

[Chibu]

And for the record, i do know that the book names (or rather numbers) and words would be repeated alot, but SQL DBs are actually much more efficient than you give them credit for. It can do a million line query in under a second, so it shouldn't be a problem.

One can convert a book from hardcopy to scanned imaged to ORC text of the book in the original format. I did it with Fields of fire once. (unless whoever's reading this is concerned with the copyright protection of the book, in which case, i have done no such thing ^-^) I do not, however know of being able to turn a pdf into a word doc. I'll see what i can come up with though.

[Bastlynn]

It's (at least in the copy of Adobe I'm looking at here) under the 'save as' options... I'll double check to make sure that's not some peculiar setup on my system.

[Bastlynn]

It's been a long time since I last posted but the code is wrapped up and going through final bug testing. I want to add administrative controls in version two before opening it up to anycomers. Yay, for getting a new job and losing your free time... I'm considering some of the hosting requests offered so far (thank you all!) but will address that once the final administrative considerations for the indexer are taken care of.

[nezumi]

There is a file like this floating around somewhere. It's basically an equipment list for everything in SR, plus a bunch of converted CP2020 stuff thrown in. It was a fan based project made like 6 years ago or somesuch, so none of the SOTA books. it's an 8 page pdf I found while trolling the internet. I may still have it but, as has been pointed out, it's firstly of dubious legality (since it lists equipment stats for everything in addition to the book reference) and includes a lot of unbalanced CP stuff. Plus, it's equipment-only and out of date to boot.

[AbNo]

*boat rocking*

Is it ready yet? :P