Speedy production dates: an alternative to Hartmann's table?

Yep, that's about the size of it. I guess what motivated me to start looking at this was to try and determine exactly how random/predictable the process was. While there have been some really encouraging results so far that indicate the process might be reasonably predictable within certain constraints, it's very early days.

One possibly interesting outcome is that we might see some change points in the data that suggest modifications of Omega's production or record-keeping processes (there's already a hint of this). So extract dates could be quite predictable for some periods and quite random for others. Or we might need separate curves or more complex curves as @eugeneandresson mentioned above. All good, geeky fun 😀

Holy smokes, that's a lot of extra data points! Thanks so much! Looking forward to adding these to the update. Indeed, thanks to everyone who has jumped in and provided data. If it turns out that we can predict production dates reasonably well using this kind of method (and I do think that's still an "if"), a web-based tool would be a great idea. Thanks for suggesting it, @kov! Let's keep in touch about that as things develop. Only time and more data will tell 😀

This is a great find! The updated data set should be ready soon (we now have n=60). Unfortunately, the data are still relatively thin on the right-hand side of the plot. But something is definitely looking fishy over there, so it's good to see there might be some clear explanations of what we should expect to see and what's driving the changes. Awesome stuff 👍

Yep, the more the merrier! Thanks to everyone who has contributed already. I have an updated data set (n=74) and model just about ready to go. Should be posted within the next 24-48hrs. We will be needing new cases to test the predictions.
Now I just have to get this pesky day job out of the way for the week... 😀

Hi all! Thanks so much to everyone who has commented and contributed data. This is why OF is such a fantastic forum! Today I'm posting an updated data set that contains 76 serial number-production date pairs for Speedmaster Professionals in the 321-861-1861 family (up from 40 in the original data set). For most of these watches, I have a photo of the extract, so we can be fairly confident in the data. I have also developed an updated model on the basis of the expanded data and things are looking reasonably good 👍 Bottom line up front: while this curve-fitting approach is nowhere near perfect, and it is still very much a work in progress, it does still seem to be a promising method.

The entire data set (n= 76) is shown in the scatterplot below...



As with the original numbers, there is a roughly linear relationship between serial numbers and production dates throughout the 60s and 70s. The slope is different on the far left (low serial numbers, early years) and far right (high serial numbers, late years) and there appears to be more spread (ie., the relationship is less predictable) in the later years. The two extreme cases from the original data set (watches with production dates in the late 1990s) can still be seen in the top right corner.

With the exception of those two watches from the mid-late 1990s, it appeared that the entire data set might be reasonably well fit by a slightly more complex curve than I had used in the original post; as was suggested by @eugeneandresson. So, I removed those two cases - effectively reducing the sample size to n=74 and restricting subsequent model development to watches made before 1990 - and then fit the remaining data with a curve. A third-order polynomial yielded the fit shown below...



This model could be considered an improvement on the original linear model in the sense that it fits the updated data set quite well (R^2=0.99) and it accounts for a wider range of production dates than the original model. It's possible though that by expanding the scope of the model, we might be sacrificing some precision. So of course the key question is, how does it perform relative to Hartmann's table?

As with the first model, I used the new model to calculate a predicted month number (starting from January 1957) for each watch and then translated that into a predicted calendar year and calendar month using slightly modified equations from the ones I originally posted (which were generally fine, but turned out to misbehave a bit around December). I then compared the model's ability to predict production year with estimates derived from Hartmann's table. Across the whole data set, the new model outperformed the table. Overall, the model predicted just a touch under 70% of production years correctly (0.68), while Hartmann's table predicted around a half of the years correctly (0.51).

Breaking things down by time, the model outperformed the table for watches produced between 1957-1969 (accuracy = 0.74 versus 0.6) and for watches produced between 1970-79 (accuracy = 0.65 versus 0.41). Both methods were pretty rubbish at predicting production years from serial numbers between 1980-89, but the model still outperformed Hartmann's table (accuracy = 0.29 versus 0.14; just two correct versus one correct respectively). As the thread that was found by @JMH76 suggests, something seems to have changed in the mid-80s and there's probably more work to do to get acceptable accuracy during that period.

The model and Hartmann's table agreed on production year about half the time (0.49). When the two methods agreed, that estimate was correct 75% of the time. When the two methods disagreed, the model was correct 60.5% of the time, Hartmann's table was correct 26.3% of the time, and both were wrong 13.2% of the time.

Regardless of the specific equation that is used, a benefit of the curve-fitting approach over the table is that it enables finer-grained prediction (eg., down to the month). So how did the new model fare in predicting production month? Across the whole data set, the average absolute error was just 3.71 months. However, the maximum error was a far-less flattering 26.74 months (!!!) for a watch with a 4481xxxx serial number that was produced in April, 1980. The updated model predicted July, 1982 for this watch; Hartmann's table did even worse, estimating that the watch was produced some time in 1983. The next worst error from the model was just under half of this. Broken down in terms of time, the average absolute errors were 2.61 months, 7.28 months, and 4.59 months for 1957-69, 1970-79, and 1980-89 respectively. Broken down in terms of references, we get the following ...



Average absolute errors from the model were generally less than 6-7 months, which is an OK outcome.

What can we conclude from all of this? Well, we shouldn't read too much into attempts to validate a model using the same data set that was used to derive it. We need new cases. Ultimately, the process might just not be quite simple enough to be characterised by a curve of this kind across all relevant time periods/references and we may have to augment the model with other information. However, on the positive side, there's nothing in the updated data that invalidates the general approach. There's probably still fruitful ground here.

In the interests of error checking and to enable folks to try this out for themselves, here are the updated calculations (where x is the serial number in the form xx.xxxxxx and z is the predicted month number):

Predicted month number (z) = (0.0142174588*(x^3))-(1.3104154822*(x^2))+(47.0857686248*x)-451.3140730771
Predicted calendar year = 1957+ROUNDDOWN((ROUND(z-1))/12,0)
Predicted calendar month = MOD(ROUND(z-1),12)+1

BREAKING NEWS: Yesterday as I was preparing this post, I was sent a list of 18 new dates and partial serial numbers, mostly from watches in the early 70s with serials in the low 30 millions. I tested the new model against these numbers and both the table and the model performed poorly. Indeed, sadly, for the first time, the table slightly outperformed the model 😵‍💫 I'm corresponding with the member who sent me the data to see if we can figure out what might be driving this poor performance.

Given the saying "All models are wrong, some models are useful", I don't think it's likely that the data are at fault here 😀

After a busy week, I've had some time to look at the data that didn't quite fit the model. Thanks very much to @Banner Roar for his help in contributing those new numbers. The manner in which the new numbers departed from the predictions of the most recent model is potentially really instructive. The plot below shows a 'zoomed-in' view of the model (blue line) and existing data set (blue squares) with @Banner Roar's data added (green diamonds). Unless you look closely, you might mistake the model for a straight line because only a small segment of the curve is shown. It's actually the third-order polynomial curve from the post above.

There are some interesting take-aways from this plot: First, the data sets corroborate one another quite well. Second, the model performs well for about half of the new observations. Third, the new data extends on a trend away from the predictions of the model around Month 200 (mid-1973) that was hinted at in the existing data. This is reflected by the slightly-right-leaning vertical column of points in the middle of the figure. Third, and perhaps most interestingly, when one looks at the actual numbers in the column, it appears that the process continued in a roughly sequential manner (ie., higher serials tend to be associated with later dates), only much, much more slowly. That's what gives it a lean. This does not look like simple random noise. It looks like a pronounced, but short-lived slowdown in production. And that kind of thing can be modelled 😀 (@Andy K, you'll recognise this kind of right-leaning column of points, right?)

Based on the existence of the slightly-later observation with a much larger serial number that falls right on the curve, it's possible that one of two things happened after the slowdown: either production sped up again significantly (ie., the curve flattened) between about January 1975 (the topmost green diamond) and April 1976 (the next blue square over to the right), or there were some serial numbers that were skipped. This is testable, which is always good!

This is not the only "Brief Lull in Production" (BLIP) 😀 shown in the data. The existing data hint at another similar event around month 300 (December 1981) for serials around 44-45xxxxxx (see the scatterplots in my post above). A zoomed-in version of that part of the existing data set is shown below. In this case, the predicted values (blue line) shoot right down the middle of the vertical column of points because they were in the data set that was used to derive the model.

Again, this column leans sequentially to the right and suggests a slowdown in production, rather than simple random noise. Based on the existence of the next point to the right after the column it's possible that production either rapidly ramped up again, or a range of serial numbers were skipped.

So how should we account for the BLIPs in making our predictions? Well, so far it looks like they are relatively few in number and relatively short lived. Also, the data seem to suggest they might have a similar shape: an accelerating curve on the front end and a relatively flat return to baseline on the back end. And importantly, the BLIPs appear to be reflective of a slowdown in production, rather than simply being the product of an increase in randomness in the process. So it may be possible to apply the underlying model to most cases, then model the BLIPs separately, using a different set of rules within those specific serial number ranges. Of course, this could also be a case of (over)fitting ad-hoc curves to a small sample that is not representative of the population! That would be bad. Food for thought 😀

G'day there! Unless I'm mistaken I think we're actually pointing in the same direction. In order to interpolate the date for a watch that was not in the original dataset, we will need to apply a rule based on what we know about the relationship between serial numbers and dates in general, right? I think this rule is what you refer to as a "preset lookup formula"? If so, then we are in luck: I have one of those 😀 Actually I have two, but the second is probably more useful than the first.

By way of example, here's a watch that was posted for sale just a few hours ago...

This watch has a serial number beginning with 3162xxxx. If we didn't have the extract, we could use the models in this thread to predict the production date. I have a little spreadsheet with the equations from this thread all set up for this...

So the earlier linear model estimates a production date for that watch of November, 1971. The updated third-order model estimates September, 1971. Because we have the extract, we know that this latter estimate is just one month off. Now, that's pretty good, but it's not perfect and there are other watches where the model misses by a large margin. So there's still work to be done.

Does this seem similar to the approach you were thinking of? Maybe I've misunderstood? I'd be very keen to hear suggestions for other ways to go about this and I'd be more than happy to post the data set so that others can have a go at developing different methods or tweaking this one. If we can improve on the table, it's all good 👍

Hey thanks! I don't recall ever having played around with VLOOKUP, but if I am understanding correctly, this could be a very nice way to go about it. Assuming I've implemented the approach properly, it certainly seems to give a good result for the new watch I posted above!! ...



Very nice indeed 👍 (at least the outcome is nice ... the implementation probably leaves a lot to be desired!)

Great suggestion! It would be interesting to know which approach to combining the results tends to yield more accurate results. So far, it seems that at least when the table and the polynomial model disagree, the model is more often correct. It will be interesting to have a good look at this linear interpolation approach in the same terms. As you say, it's bound to have its pros and cons.

Awesome stuff! Still lots of details to lock in, but one way or another it's starting to look like the table is TOAST! 😁