JFK Timeline Project

New JFK Timeline Project has been released. Deserves special mention here (apart from how useful it's going to be) because ROKC's very own Steven Duffy had a big part in making it. He gave years of his hard work from his index to the Project. Bookmark this puppy and use it lots!

http://www.jfk-timeline.org/index.php

Frankie Vegas wrote:New JFK Timeline Project has been released. Deserves special mention here (apart from how useful it's going to be) because ROKC's very own Steven Duffy had a big part in making it. He gave years of his hard work from his index to the Project. Bookmark this puppy and use it lots!

http://www.jfk-timeline.org/index.php

Yeppers.

Much kudos to Brian and to Steve. I hope it is used. Lots. When I get a chance, I intend having a play around with it.

Hi all! Haven't been up here as much as I'd like, been nose-to-the-grindstone with data entry and such.

I'll just post a few things in this thread, hopefully people will see it.

First of all, thank you for your encouragement, and your support, and your kind words! This is a bit of a bigger enchilada than I imagined at first, and as a n00b fledgling JFK researchers I find myself being dragged kicking and screaming into the world of the data, and the controversies around the data, and the personalities around the controversies around the data... it's makin' my head spin a bit, at the moment, to be honest - 'cause I'm trying "real hard" to focus on the actual work.

We were a rag-tag team of six at one point, I found a buncha college students who could type and needed money, and my vision for this web site.... how can I say this... I figured "I could seed it, and you guys could organize it", kinda thing. I mean, I initially viewed "my job" as simply posting the Events and the People and the Evidence, and I figured that would be "enough" to get people to jump in and start working with the data, organizing it, putting it together.....

Because, "I" can do that, but then you guys get "my" data organization, which is not necessarily what you want - you want to work with "your own" theories, which means "your own" data organization, right? ('Cause I mean, you don't know me, maybe I'm one of those wild conspiracy types who thinks the mob and the CIA and Lyndon Johnson ALL did it.... yuk  )

At least that's the way I was lookin' at it around July 4th when we "opened" the web site, but since then, it seems the reaction has been mainly "where's the beef" - like, there's a small crowd of genuine research types who've signed up on the site, but it seems they're waiting for me to deliver something more impressive.... or something.... and I understand that, 'cause genuine researchers have seen all this data before, right? There's nothing "new" on the jfk-timeline.org site (except for the math) - all that data already existed, it's not like I'm out there interviewing new people or anything (yet). So, the general take so far is that the site hasn't really lived up to the expectation, and I didn't help that perception either, 'cause I created the Help pages before stuff actually worked, so there's a lot of wonderfully flowerly language that describes stuff that's still being put together.

The thing I'd like to communicate, is this is a LOT of work - it's a heckuva lot more work than I though it would be.

For instance - right at this moment I'm building out the stuff around Jack Ruby. Jack is a "complex" character, relatively speaking, in the world of JFK data in general - but he's not the "most" complex (Oswald, for instance, is more complex, 'cause he has people running around trying to impersonate him all the time, which Ruby doesn't seem to have - but nevertheless he's frequently in "two place at once" and stuff like that) -

And this is exactly what the computer does, is notify you about stuff like that, and give you a lot of information about the particulars - but until the DATA is entered correctly and organized correctly, none of that lovely math is going to work (or let's say, the results will be mostly meaningless).

So, I had to pick a place to start, so I picked Jack Ruby. There's a kinda half-together model of the Tippit shooting in place, and just getting started on a bucket of Events related to "Oswald in New Orleans" - but other than that there's no hierarchical data organization - all there is so far is Events, People, and Evidence.

I was kinda figuring people would want to jump in and organize stuff for themselves, but maybe it's kind of a been-there-done-that thing, and I mean, I'm a n00b so I'm perfectly willing to jump in and go through the exercise and do all the work myself (and I'll undoubtedly become somewhat of an expert in the process) - it'll just take a while, that's all. I've allocated two more weeks for some rudimentary bucket-building, and after that I'll spend another week on the niceties of the graphics, the timeline displays and all that, and then I have to resume some kind of gainful employment - but in the meantime I'm full time on bucket-building, and so if you check the site from one day to the next you can see an increasing amount of structure and organization around Jack Ruby.

For instance I'm adding a lot of missing people right now, Ruby's friends and his mob connections for example, and the deltas between his story and the stories of, say, Beverly Oliver about the night of 11/21, and the whole deal with Thomas Eli Davis III, and stuff like that.... so, "building out the data" so eventually the web site will tell the story - you push the Jack Ruby button and you get the whole story, just like it says on the Help pages.

So, that's where I'm at. So far so good. The computer is revealing some very interesting things already! (It starts with the obvious stuff, just like I do - so, window crossings - events that can not possibly have occurred together because the same person would have to be in two places at once - that kind of thing).

Cheers - Brian

Its worth the waiting, Brian. I am sure everyone will appreciate your efforts with limitless patience.

Brian, it's an awesome resource and we are grateful. Please ignore the ones who want everything and want it now, and sorry more haven't been willing to jump in and help (it can be a tough crowd to motivate) (let me know if I can help in any way).
For a while I was organising protests. I would set it up, hire the PA and get speakers and liaise with the council to not get kicked out of the space we wanted to use, the police to make sure we would be safe and to shut down the main rds if we wanted to march etc etc. I never realised how much work went into it. Anyway, after the rally there were always ALWAYS  a bunch of folk who were nowhere to be found when we were organising that all of the sudden had an opinion on what we didn't do right, how it could be better and what they would have done.
You have to ignore those types people, unless they are genuinely helpful or constructive then they can sit down and shut up lol.
All the best!

Frankie,

My wife has a poster over her desk to remind her what to expect whenever she takes on a new project:

The Six Phases of a Project

1 - Enthusiasm
2 - Disillusionment
3 - Panic
4 - Search for the Guilty
5 - Punishment of the Innocent
6 - Praise and Honors for the Non-Participants.

Kudos to you for taking on such things.
You are far braver than I.

Brian,

I can only speak for myself. I think what you have done is enormous. Absolutely and without doubt, a boon to research. Your result with a time for the Tippit shooting showed what this is capable of (not that the time was necessarily correct, but with all the relevant data input, it should at least come out with an answer that makes sense of the data.  

That's all on the pro side.

On the con side... I just cannot get my head around how to use it - even the navigation takes me ages to figure out when I go there. Everything I do on the web is of the "for dummies" variety. 

I mean, for me, it's like looking at Apollo 11... I can appreciate the beauty of it, the work that went it and what it's potential is... but actually OPERATE it???   

I know it is way too much to expect you to simplify it so I will persevere trying to teach myself... it would be a tremendous pity if this resource remains under-utilized.

terlin wrote:Frankie,

My wife has a poster over her desk to remind her what to expect whenever she takes on a new project:

The Six Phases of a Project

1 - Enthusiasm
2 - Disillusionment
3 - Panic
4 - Search for the Guilty
5 - Punishment of the Innocent
6 - Praise and Honors for the Non-Participants.

Kudos to you for taking on such things.
You are far braver than I.

She is a wise woman. The poster is correct

Greg, I am like you and will probably pick up how to use the timeline once others ahve and can explain it to me in simple language.

I had a go of it last week.  Once I'm comfortable I'll post a step by step.

Hello Von Hello wrote:I had a go of it last week.  Once I'm comfortable I'll post a step by step.

Excellent! Thanks Lee.

Cheers, Lee!

Colonel Von Hello wrote:I had a go of it last week.  Once I'm comfortable I'll post a step by step.

Thanks Colonel, I'm looking forward to hearing your experiences. I'm guessing that as an early user you probably encountered a few hardships, yes? (Like, maybe, a lot of the stuff wasn't working yet?)

I haven't gotten onto Jeff's site to read his collected comments, but will do so shortly. I'd like you to know though, that I'm learning how to use the system at the same time you are. I'm doing "actual" research on it, for the first time - and so far so good. As a starting point I picked Jack Ruby, and that's the area of the data that's being built out right now. You can watch as we build it, the buckets are being built and there's already a public timeline there, and over the next few days the timeline will fill up and it'll get a lot more detailed.

How I do this is, I work mainly as a System Administrator, so the ordinary Users can't see the bugs while I'm developing stuff - then once I'm reasonably sure things are working, I "expose" them to all users (and usually that involves reducing functionality by privilege level, instead of building more). That's why you sometimes read stuff in the Help pages before the buttons appear on the screens - I write up the way I'd like it to work, and then I make it work that way.

And, this is my first time as a JFK researcher, so I'm learning. Right now the system is slow and cumbersome, difficult to work with when you have to do data entry in volume. I don't want to have to push 1000 buttons to get upload data - so this morning we built a "Reconcile" function that automagically links your Events and People, so you don't have to build all those buckets by hand. (I actually did build them by hand at one point, before getting wise - but this is the type of thing that's going on, trying to look at the work flows and making them efficient, quick, and easy).

Meanwhile in the process of entering Jack Ruby's data points, I discovered people who were missing, and all kinds of other "ancillary" stuff that requires attention "during" the data entry task - and for that purpose I've found it useful to bring up a second session, so I can do Events on one screen and People on the other, and just alt-tab between them.

So far so good, all that is working, and even better than expected at this point - and the next step is to pay attention to some of the visualization stuff - FOR example - there are two divergent versions of Jack Ruby's events on the night of 11/21/63. In the one version he has dinner with Ralph Paul then goes over to the Cabana Motel where he meets Lawrence Meyers around 11pm, and only stays for a few minutes before making the excuse that he has to get back to his club and attend to business. In the OTHER version he has a steak dinner with Lawrence Meyers at Campisi's Egyptian Lounge, and then he goes over to the Cabana where he remains until after 2am. Oh and while they're at the Egyptian, Lawrence and Ruby retire to Campisi's private office where they make a phone call, and immediately after that call they take off for the Cabana.

Jim Braden and Morgan Brown are at the Cabana that evening. Chauncey Holt (and Moceri and Nicoletti) are supposed to be there but they had car trouble so they're late. And they were supposed to meet Herminio Diaz, so he's probably hanging around the Cabana someplace too. Quite the party, it seems.

So, when I'm displaying these two "alternate" story lines on the Timeline, I'd like to see the one version in green, and the other in red, so I can quickly and easily distinguish them and determine which events belong to which story line.

So that's what's going on now - iterations between the bucket-building and the inspection of the resulting timeline.

Things are substantially better than they were last week, there are still bugs (like the Alias screen isn't working quite right yet), but by and large you can do what you need to do. It's cumbersome and a bit tedious at the moment, but we're working diligently on that piece, and by this time next week it'll be a lot quicker and a lot easier.

For those interested (or even trying to keep up) I'll post periodic milestones to keep you informed and link you to some results.

Right now I'm working on Jack Ruby. At the moment I have most of the events loaded between 11/21 and 11/25, and I've linked them to the underlying people.

So right now, when you push the "Events" button next to Jack Ruby's name, you get an actual timeline, and you can see that here: http://www.jfk-timeline.org/timelines/213.htm

Now that I have some Buckets in place, you can see if you look at the actual records that all the Evidence is still "embedded", it's inside the People and Event descriptions and it has to be pushed down into actual Evidence records.

So, y'know, if you go to Jack Ruby's profile and read the Description you'll see CE-1713 and stuff like that - and the CE-1713 Evidence is already in the system, but it's not linked to Jack via an Evidence bucket. That's the next step - doing the same thing for the Evidence, that's just been done for the Events.

Once the Timeline exists, the next indicated milestone is to put all the information into a Model, and let the system start playing with it - but before that happens all the Evidence buckets have to be properly linked. And that's a big job (I picked Jack as a starting point specifically for that reason, 'cause he gets a lot of coverage system-wide), it'll take two or three days.

What the correctly structured Evidence does for us, is it lets us start working with the Confidence Levels. As I mentioned to Greg in another post, there's a "system" around those Confidence Levels, they're not just SWAG's (even though they may start out that way, and that's perfectly okay - the computer will fix them for us eventually).

So like, right now, I'm posting a lot of stuff about Jack Ruby because that's what I'm lookin' at with all this data - and I started with a window of a few days each way, around the assassination - so, from the 21st in the morning through the 25th, 'cause that's where I could easily get lots of information, there were already hundreds of events in list form. And now that I'm convinced we can get from the Events list to a Timeline just by pushing a couple of buttons, I can justify taking the time to load the "rest" of the data about Jack Ruby, so like, the early days of his life from 1930-ish onwards, the later part around his trial and all that -

And in the process I've discovered that it would be very helpful to get some "life histories" into the system, in addition to Oswald's and Jack Ruby's - for instance Crafard, that's an interesting one - Eva.... like that. Eva is a real piece of work! I was reading about her last night, there's Eva's landlady on record who says she moved in sometime in '61, and all since then Ruby barely came by but once a month, and most of the time when he did he just honked the horn and sat outside in the car till Eva came out. That seemed to change when Eva became ill and went to the hospital, then and afterwards Jack seemed to come by more frequently and spend more time.

Jack seems like a halfway decent guy so far, he musta really freaked out when he found out what they wanted him to do. I don't believe for a micro-second that his shooting Oswald was a "spontaneous act". Not when you're stalking the DPD for three days in a row with a loaded pistol, and going out of your way to generate alibis for that activity. (Ruby didn't mention that at all in his first testimony, and he also didn't mention anything about the party at the Cabana Motel on the night of the 21st). Ruby seems to have quite a few people generating alibis for him, that part is really interesting, and it's one of the parts the computer is definitely going to help with.

So, Evidence here we come. Next step is to link the Evidence Buckets, and then the Timeline will start having numbers attached to it. Once that happens we're off into modeling-land and we can start juggling the numbers to see what makes sense and what doesn't. You can tell a lot of things by inspection, just by looking at the Timeline, but the computer does a much better job with the details. Like for instance, one of the interesting events is Ruby's encounter with Kay Coleman and Harry Olsen. At first, when that Evidence starts out, Johnny Simpson is saying "over an hour" and Kay Coleman is saying "over an hour, maybe even two hours", and Harry Olsen is saying "two or three hours", and by the time all the Evidence is in place that's been whittled down to "just a few minutes - maybe ten minutes or so". Sounds very much like Ruby got Simpson, Coleman, and Olsen to alibi for him... or else, why would all THREE of them come up with the extended time, all at once? The probability of that happening in real life is very low, what would ordinarily happen is one of them would say "an hour", the other would say "half an hour", and the third would say "five minutes or less". There's no reason "all three" of them would multiply the actual time by ten, is there? (Unless they were "all" taking Preludin? Is that possible? Yuk.  )

I have no idea whatsoever about buckets and swags and such like but I am still very hyped about this. Thanks for the update! I can't wait to play around with it and teach myself bit by bit how to use it.

Frankie Vegas wrote:I have no idea whatsoever about buckets and swags and such like but I am still very hyped about this. Thanks for the update! I can't wait to play around with it and teach myself bit by bit how to use it.

I have an idea.

But before I tell you about it, here is the third iteration of the Jack Ruby timeline:

http://www.jfk-timeline.org/timelines/216.htm

It should have most events up till 11/26 or so, but the trial events aren't in yet. I was hoping to have them done by today but got sidetracked, so they'll be ready in a day or two. Meanwhile several hundred more evidence links have been fixed, so... a little bit at a time.

I would love for you (all) to scrutinize the early part of the Jack Ruby timeline at this point, as near as I can tell I've scraped most of the "official" events and several unofficial ones as well, there's 512 events total on this timeline, ranging from the first one in 1940 to about 11/25/63. I would greatly appreciate any guidance on what's missing (and I'm sure there's LOTS of stuff missing, probably even stuff on old threads right here on the forum). I don't claim completeness, but with your help maybe we can get there.

Which brings me to my idea.

I would love for a few of y'all to jump in and test the water on this site - how can I say this... "the level of work is overwhelming". It's taken me a week to load the Jack Ruby events, and there's so many more I can't even begin to count - but that's not even the issue, the issue is they need to be ORGANIZED.

You can see what's going on in the timeline there, for most events you get a "range" of times, there are several reports of each event, and they come with different times, but when you look at the timeline you can see the cluster, and you can say to yourself, "okay, the event happened in this time frame more or less" with a reasonable degree of confidence - that is to say, it's an instant intuitive visual. Here's all these data points, they all happen with this range, they corroborate each other at least "that" much, right?

Another interesting thing you can see on the timeline is some of the patterns of the alibis - you have to stretch your mind a little because of the sea of blue dots, but if you pick a set of events that you're reasonably sure is an alibi (like say, that initial set of stories about the early morning meeting between Ruby and the cop and the stripper), you can look at the related events and the events around it, and some of them have a "different" pattern than ordinary normal events, right?

So, here's what I was thinking - the primary need is to ORGANIZE the data. Some of these events are actually the SAME event (being reported by different people, with maybe slightly different times attached to them), and some of them are RELATED events, either they're events in a sequence that form a larger event (like the Tippit shooting), or they're all "part of a conspiracy" or whatever (like for instance Loran Hall's visit to Silvia Odio with Lawrence what's-his-name... Howard? and William Seymour - that one is REALLY interesting!) -

So anyway, some of y'all (many, probably) are "subject matter experts", you have "areas of expertise", stuff that you probably know more about than anyone else on the planet. Your "favorite" areas of research, let's say. And, if you're willing, I'd be very happy to grant you instant Research privileges so you can modify the data and load stuff and make it public (just exactly like I'm doing now - but you don't have to be doing a thousand records a day, it would be better to take one's time and do one event and do it right, 'cause the quality of the data is important). And, it's a great way to test the waters, and I'll support you during the learning curve if you wish (which is really very quick, it's not hard at all once you get the hang of it, it just takes a little time the first couple of times).

The more data that's being loaded, the more it seems is missing. And I haven't been around that long, I'm not aware of all the discussions that have taken place and all the information that's been exchanged. So like, I can continue working on the "official record", 'cause that's a very tedious task and I can probably do it in the background while other things are going on, but maybe I can get a few of you to start looking at the scope of what's there, and making suggestions about what needs to go in there (and where to get the information)? That would be superb! That's an open offer, any time you'd like to jump in is fine.

Meanwhile, here's this Ruby timeline and it's probably worth a glance at this point, as near as I can tell my coverage of the "official" events up until around 11/25/63 should be pretty good and I'd appreciate a little honest feedback as to whether that's an accurate perception or not.

I'm keen to help, although I'm more of an activist than a details person. I have just made an account on the site (the confirmation email went to my 'junk' folder in my email).
I went to the home page and searched first 'Yates' as I am following the discussion here about him. It went to a page that had information about him and then there were links to buckets (although they haven't been filled in yet?) so I went back and searched 'Ruby'.
I clicked on Jack Ruby, went to his profile and then down the bottom of the profile clicked on 'event bucket' as I thought it would take me to the page you linked above.
I ended up on a page that was called 'event bucket editor'. Not sure where I took a wrong turn?
But maybe by my signing in and trying to find the page you can see the level I am at (which is probably around -2 lol).
I will spend the next few hours messing around until I feel more comfortable navigating the site.

I'm still getting to grips with it.  It looks very interesting indeed although it is going to take me a while getting under the hood.

There is much already populated in the timeline. For instance, Oswald moving to NOLA brings up a timelines like this:



Each blue dot being a timed event.  Clicking on it will bring up an event profile page like this:



Brian, out of interest how is the confidence level of an event assigned?

Frankie Vegas wrote:I'm keen to help, although I'm more of an activist than a details person. I have just made an account on the site (the confirmation email went to my 'junk' folder in my email).
I went to the home page and searched first 'Yates' as I am following the discussion here about him. It went to a page that had information about him and then there were links to buckets (although they haven't been filled in yet?) so I went back and searched 'Ruby'.
I clicked on Jack Ruby, went to his profile and then down the bottom of the profile clicked on 'event bucket' as I thought it would take me to the page you linked above.
I ended up on a page that was called 'event bucket editor'. Not sure where I took a wrong turn?
But maybe by my signing in and trying to find the page you can see the level I am at (which is probably around -2 lol).
I will spend the next few hours messing around until I feel more comfortable navigating the site.

Hi Frankie - ah, okay. There's two different things going on here. (You tried the "other" one).

How the Jack Ruby timeline was created, is, to the right of Jack Ruby's name on the Manage screen you should see a button that says Events. If you push that, you'll see the list. What this button does, is it aggregates all the People buckets underneath each Event. That's one "direction" of cross reference, so for example, inside the event "Jack Ruby shoots Oswald" there is a People Bucket, and inside that you'll find Jack Ruby's name, and Oswald's name. When the Events button is pushed next to Jack Ruby's name, the "JR Shoots Oswald" event shows up "because" Ruby's name is in the People bucket. So, we're just aggregating all the places where JR appears in the People Buckets.

The "other" direction of cross-reference, is the one you tried, and that is, directly underneath Jack Ruby's profile there is an "Event Bucket", and yes you are right, that should contain the entire list of Events that JR was involved in. However I have not yet organized this direction of cross-referencing, for the Jack Ruby events. (That's because I'm still not done entering them, there's about 200 more to go).

This issue of keeping the cross-references in sync, is addressed by "another" one of the buttons near Jack Ruby's name, it's the one called "Reconcile". That feature is still in a half-working state, but it should work okay for Jack I think. At any given moment, if you push that, it'll show you several things: a ) you'll see all the places the name "Ruby" appears in the Event Name, but the Profile either doesn't yet have a People Bucket, or the one that's there is empty - then below that b ) you'll see the places where the Person is in the Event bucket but the Event isn't in the Person bucket, or vice versa.

Lemme see... the other reason I haven't yet filled in the Event bucket for Jack, is kind of back-door and sly: we already HAVE the bucket of Events, we built it when we pushed the Events button next to Jack Ruby's name. (Right at the bottom of the list there's a link or a button that says "Build A Bucket With These Events", push that and on the next screen you get to name the bucket, hit ok and you'll get the bucket id). So, since we already have the bucket, we don't have to build it again - what we can do is LINK it directly underneath Jack's Profile, "as" the Event bucket. To do that, we have to do exactly as you did - go to the Profile screen and push the Event Bucket button, and if it says "Create" when you push it, on the next screen it'll ask you whether you want to "link new" or "link to existing", and in this case you choose EXISTING instead of new, and you give it your bucket name, and you're done. Now you have a bucket full of the proper Events as Ruby's Event Bucket. (IF the Profile button doesn't say "Create" it means there's some other Event Bucket already there, so you can either get rid of it by unlinking it, or you can combine the bucket contents and clean up the leftover empty bucket when you're done).

Colonel Von Hello wrote:I'm still getting to grips with it.  It looks very interesting indeed although it is going to take me a while getting under the hood.

There is much already populated in the timeline. For instance, Oswald moving to NOLA brings up a timelines like this:



Each blue dot being a timed event.  Clicking on it will bring up an event profile page like this:



Brian, out of interest how is the confidence level of an event assigned?

Hi Colonel - oh yeah, the NOLA timeline - that was the beginning of an experimental thing I was working on just to prove we could create an Event bucket and use it to build a Timeline. (You can tell it's kind of sparse, it was from a while ago). Hopefully we'll have a better timeline there pretty soon.

The confidence levels, that's a great question. It's a long answer. "That's where the math comes in", is probably the short version. Let's see - initially, a SWAG will work, you don't want it to be too far off though, because that's what the optimizer looks at. There's different "flavors" of confidence, it means something different for an Event than it does for a Person. For an Event, the confidence is determined by the variability of the optimized Event Window relative to the stated Window. And that, is somewhat complex, ... um... (fishing for words)... The confidence level is something the system optimizes. "It" eventually figures out how confident "it" is in the Events it's looking at. You can initialize the confidence with a SWAG, but during the process of optimization it's supposed to "tend" and "trend" towards its natural (and proper, and correct) value. How quickly or slowly that happens depends on how many data points and how many iterations and so on, but this is one of the ways you can ask the system questions. And all of this, gets into the advanced area of "modeling", which is very much where we're headed with all this.

You'll notice right now most of the confidence levels are 0. Some are non-zero but I'm using those for another purpose right now, I've not yet initialized any "real" confidence levels. We don't have to do that till we start modeling, which is still a ways away. (Couple of weeks? Three maybe?) We can build timelines without confidence levels. Y'know... maybe I can reactivate one of the old models, just to show you how this works. We had a Tippit Shooting one that was working pretty good, I kinda dismantled some of that stuff (or at least left it in a halfway state while trying to clean it up), but lemme see if I can find it and make it work (if I can I'll post back in the morning). What you can see is the system "optimizing an event" and "assigning a confidence level to it". The model output is something like this: "JD Tippit got shot at 1963-11-22 13:09:53 +/- 47 seconds, with a confidence level of 82%". And that kind of output occurs "after several iterations" of an optimization experiment - so then, what you can do is simply leave that number in place, and thereafter the system can use that number when the same Event "isn't" being optimized (in other words, when it's serving as a reference point for some other Event that "is" being optimized). I dunno if that makes sense, hopefully it does, we can talk more about it for sure.

Colonel Von Hello wrote:I'm still getting to grips with it.  It looks very interesting indeed although it is going to take me a while getting under the hood.

Brian, out of interest how is the confidence level of an event assigned?

Hi Colonel - this is a great question you're asking here, and it deserves a better answer.

Let me amplify quickly (everything I said before is true, but here's more):

Specifically for an Event - there is an interplay between the event "times" and the "evidence" that supports the event.

Here's how it works: in most instances, an Event is "defined by" the Evidence. In most cases (at least here in the JFK world), we don't have actual physical evidence - all we have is "testimony", or some kind of "official record" perhaps, or maybe a document, y'know, a newspaper clipping or something. AND, we have interviews, which may be on YouTube, which are also "testimony" in a way. In other words, for the real REAL description of the Event, we have to rely on peoples' word - their memories, their motives, their accuracy.

So, part of this, is, when we define an Event we have to be honest about assigning variances to it. You'll notice in the Event Profile that all the times have "variances" attached to them (including the event duration, which also has a variance). That's kind of a more sophisticated concept than it sounds - because you know about ordinary statistical variance, but this variance is more than that. And not only is there just the ordinary variance being represented in the Profile, there are different "versions" of variance - and you can use that for instance when you've got an event like the late night encounter between Hansen and Ruby, where you're pretty well sync'd up on the "starting time", but the "ending time" varies all over the map - at first the testimony says "two or three hours", and by the end of the HSCA it's down to "just a few minutes, maybe ten or fifteen".

What we're doing is defining a Window within which the Event "may have occurred", and notice the word "may", that's also part of the confidence level, and obviously that's where the Evidence comes in, and our confidence level in THAT. Confidence levels for Evidence are more straightforward, conceptually they're a lot closer to what we ordinarily conceive of as everyday confidence (or lack thereof). And the Evidence in turn, "sometimes", relates back to the confidence levels in the Person records of the people giving the testimony.

SO - when the system calculates the confidence for an Event, it's looking at a lot of things. There's a pretty long formula for how it actually calculates the number, and the reason it's done this particular way is probably "theoretical", it has to do with the Cost Function for Optimization (which you can read about if you're really interested, it's math though, real math with integrals and derivatives and lottsa little symbols, ... but you can check Ising Model, Boltzmann Model, Black-Scholes Model, all of that is somewhat along the same lines of what we're doing).

During optimization, the system "juggles" the Event Windows. That's how it really works. It does something very similar to the pebble-sifting analogy I posted earlier. What it's doing is kind of "sifting" the windows, through different sized analytical mesh, and kind of how it works relative to the Cost Function is like this: you begin with pebbles of all different sizes, and then you use increasing mesh size in your chicken wire grid, to separate the pebbles by size - so you end up with "buckets" of event windows where bucket A has all the small ones, bucket B has all the medium ones, that kind of thing. And then it calculates the Cost Function based on how "expensive" the interactions between the windows are. And that in turn is based on the magic behind-the-scenes "almost AI" system that analyzes "event logic", for example the system knows that the same person can't be in two places at once so it assigns "expense" to that kind of a window interaction.

So, what happens is, "confidence levels" end up rippling upwards into these buckets, where there is then a calculation that is kind of "global" in nature (you can look at it almost like the "temperature" in a gas), whereupon the number then ripples backwards back down into the original events. This is what happens during an "iteration" of optimization. Little by little, the system as whole moves along a "cost surface", almost like a marble rolling down a hill or something. The "optimal" window configuration is the one that results in the least cost. And the "amount" of that cost is what ripples back ultimately into the confidence levels in the leaf nodes.

I know it sounds a bit technical, but once you've done a little work with the screens you'll "grok" exactly what's happening, it becomes intuitive at some point. To begin with, like right now at this very early stage when we're just doing data entry, the system will help us with the initial SWAG's - for instance there's a set of rules you can invoke that goes something like, "if there's one eyewitness it's a 25% initial confidence, if there's two it becomes 40% except if those two people are related, in which case it drops to 35%"... stuff like that. When we get to that, I'll show you how that works (you just press a button, basically, during the part of the workflow called "Bucket Validation").

By the way - once you have a bucket full of Events, you can try Exploding it, that will show you very quickly in what condition the underlying cross-references are. Right now for example, if you explode the Jack Ruby Public Third Effort or whatever it's called there, you'll see that there are plenty of Events, with a few People underneath them, but there is no Evidence yet. That's where things stand at the moment - the Evidence hasn't been connected yet. It's "there", but the cross references aren't built yet (in other words, the "Evidence Buckets" either don't exist or have nothing in them). And, I intend to start working on that this evening, so in a day or two you'll start seeing Evidence records there, when you Explode Jack Ruby. (Ha ha, that sounds like fun! )

I dunno... hopefully this explanation helped and you're not more confused now than you were before. The method we're using is called "constrained stochastic optimization", it's a time-honored Nobel-prize-winning big-time-money-making very successful method, it's been applied in many areas already. All it really is, is a probabilistic version of "curve fitting". We're just threading the right curve through the data (or trying to, anyway).

Okay - I installed the filter into the Timeline Editor, should make this much easier. When you go to load the timeline (Operations => Timelines => Manage Timelines, and then push the "Contents" buttons next to the timeline of your choice), you'll see a "LONG" list of events on the left hand side. And, underneath it, there's a little text box and a button that says "Filter".

Let's I want to see "just" the Event names beginning with "Jack Ruby", I can make that left-hand list a little more manageable (and quicker to display). I just type "Jack" into the text box, and then hit the Filter button. Now I'm seeing only the names beginning with "Jack".

Keep in mind that these are actually Hash Tags you're looking at (in the list), and not real Events. So, the names are compressed - for example "Jack Ruby" becomes "JackRuby". This is good though, because let's say I have events beginning with "Jack Ruby" and "Jack Lawrence" and "Jack Valenti" or whatever - and I only want to see "Jack Ruby", so I can type "JackR" into the filter field and that'll work.

You can play around with it, I tried it via the front end and it seems to work fine. This is the same mechanism I use to load Buckets, so for instance if I want to load a People bucket with Ruby, Oswald, and Loran Hall, I first type "RubyJ" into the filter field (and click him over), then "OswaldL" (and click him over), and "HallL" (and click him over), then I'm done. This way I don't have to wait for gazillions of Hash Tags to load all the time, plus it helps focus my mind when I'm trying to build a bucket with multiple items in it. Try it, I think you'll like it.

Other data types - buildings and organizations are naturally "People" (since they're reasonably permanent and have "life-spans" and all that). So, you'll find JURE and Alpha-66 under "People".

"Conditions" are naturally Hash Tags. So for instance, let's say we have a law that expires on 1/1/63, and gets replaced by another law - so, "Changing Conditions", right?

You "could" represent that as Events, but it's not really "natural" that way. However you can easily create a Hash Tag called "AB-563" or whatever the name of the law is, and drop that Hash Tag wherever it has to go.

For instance - in relation to the possible involvement of Oswald in any early "spy rings" in the housing projects or whatever - there are "events" in that story, and there are "conditions", and after looking at it for a few moments it should be pretty natural to figure out which is which, yes?

Right now I'm grappling with the first end-to-end Evidence buildup, that's the Silvia Odio visit as we discussed - and I "just now" found out about Bernardo de Torres and all that - so it turns out this is a PERFECT set of scenarios for a little modeling exercise.

The problem here, is the Evidence is "really weird". It's not so much the presence or absence, it's the way it's structured. Also, this is one of the MOST "hot" areas/topics for disinformation, it seems. (Y'know, like how DeTorres got into the Garrison investigation in the early days and kind of "distracted" that for a while).

So like, in this very simple story which takes all of two or three days in real time, we have "related stories" that encompass HUGE time frames. And, we also have competing models, for instance there's the "Loran Hall and William Seymour" theory, and there's the "Bernardo de Torres and Edwin Wilson" theory (which by the way, doesn't dovetail 100% with the evidence either), and they may even come together at some point through General Walker (or his batman perhaps, however that works).

Anyway, "Jack Ruby timeline almost done" as of this morning. There's a Loran Hall there now, and a Crafard, they're sparse ("Version 1") but they're there, they were built in exactly the same way as the Jack Ruby one. By the end of the day there will be a Silvia Odio prototype too.

Hi all - quick word: I've noticed there's a lot of seasoned veterans on this forum. Some of y'all are probably writing books 'n' stuff like that. So, I figure you may be interested in the "dissemination of information" (yours in particular, maybe - or just in general).

If so, you're watching this n00b in action. You can check the web site once in a while, and inspect the data relative to the way it looked the last time. Right now for instance, you're seeing me "drilling down". This is me entering data, right? So, I told you the way I do it, "terrain mapping". How that works in practice is, I do the "Jack Ruby First Effort", then I look at that and slap my forehead 'cause of all the easily identifiable gaps - then I go back and do more research, and try to fill in those gaps, which then results in the Second Effort, and so on. Iterative drill-down.

And, I stop when I have "enough" detail (for the time being), or, when something gets "uninteresting". For example, right at this present moment I'm going through all the LHO events putting the People buckets in there. I'm up to 1-1-63 just now. (If you hit the Events button next to LHO right now, you'll see everything that begins alphabetically with Oswald till 1/1/63, but not yet stuff that begins alphabetically with LHO, haven't got there yet). The bulk of the events are yet to come (which tells I need to go back and get some more data on the early years).

Oh and, in addition to the alphabetically "Oswald" events till 1/1/63, you'll also see some events (about 70) related to "Oswald in New Orleans", which we'd done previously as a demo, but that was an old bucket and all those will be covered in the sweep that's presently occurring - so by the end of the day there should be a nice juicy Oswald First Effort timeline in place. (I found I have to do this "before" I can complete the Ruby timeline, there's some People in particular that I had to add, so the Events had to come first, and yadda yadda).

So, this is how my n00b mind works, yes? As I'm reading something (anything - your book maybe), I'm basically reading about "Events". And, while I'm reading, my mind tries to identify the People around those events, and it starts to kind of "model" those people, based on the information from the event stream. Then, if a person becomes interesting, you go back and do more research, which means you're "drilling down" - you want more detail - both on the Events and anything else you can get, any kind of evidence at all - even oddball stories, 'cause they may generate useful leads.

So, this is the "model of learning" that's being (hopefully) created, and hopefully it gets a little assist from the mathematical piece eventually. The database should be very natural though, "intuitive". You got People, Events, and Evidence, and your job is to cross-reference them. ("That's what we do", right?) Then when you have "enough" cross-references, you can begin modeling stuff - which means considering theories, "juggling" events, deciding for yourself how much confidence you have in what you're reading -

Some of the stuff I read, is pretty persuasive. Y'know, coming from a strong personality like a Garrison maybe, he sounds... "authoritative". But here, where the pedal hits the metal is when you're connecting the Evidence records. Because then, you have to be precise. The system FORCES you to go all the way down to the original sources of the information. (If you don't have that, you can't do much of anything useful, and the system will tell you "don't bother"). You're fully aware I'm sure, of how much regurgitation there is on the net and in books and just among people in general - and some people weave a great story but they don't have many references... that kind of thing. At some point the system forces me to be "diligent", and right now when I'm just entering a list of Events or an occasional new Person it's pretty loose, but the minute I start connecting those Evidence records, wham, I can't afford to wear a loose hat anymore.

(It's kinda elegant how it works actually - pats self on back  - it draws you in and then it hits you with a little rigor! - I figure maybe it'll help the raise the overall quality of the space eventually).   

So y'know, "you're seeing the loose part right now". The minute you see those Evidence records going in, you'll see the whole thing tighten up. Right now we're "filling space", trying to get "coverage", and once we clean up the mess we're going to have a real nice surface - nice and smooth. Then it'll be a lot easier to fathom the whole thing, and if you come up with an exciting new area of research it'll be pretty easy to figure out how and where it fits in with the bigger picture.

Oh yeah - if you go to the People Menu, and select "Search People", and in the field where it says Creation Date:

type:

">2014-07-21 00:00:00"

just like that, without the double quotes

That being yesterday's date, you'll see the new People records I entered yesterday and today. Some are just placeholders, others already have information.

Well, there's an LHO timeline up on the site, it has much (most) of the "official" record. Doesn't have a whole lot beyond Garrison time frame (yet), but, there's enough of a landscape there to where drill-down can begin.

You're the experts, what do you think? There are definitely times and events and people/places that need to be filled in - like the Kerry Thornley stuff, the Odio visit, the childhood, ... there's a lot of missing detail, but at least you can see where it belongs (on the timeline, hopefully, at this point).

So, I did promise the Odio end-to-end, and I'm working on that. There was some missing stuff I had to find out about (De Torres and what that all means), but it's getting there.

Now it's down to the Evidence in both cases, and that's still going to take a minute, there are placeholder records without links right now, and it'll take a couple days to get through all ten thousand of those. But we're very close to fun and games.

Meanwhile any input you may have on the LHO and Ruby timelines is greatly appreciated!