The year was 2004. Your author, beardless (and much younger), was about to embark on an ongoing adventure of 21 years so far. In room 85 of the University of Edinburgh School of Chemistry I sat down next to James Davidson, my predecessor on a molecular modelling PhD project supervised by Peter Tasker and Simon Parsons. Having started my PhD in October of that year, I was to learn, for the first time, how to refine crystal structures. For context for younger readers, neither Olex2 nor Shelxle existed yet. Choices and were more limited, WinGX, Crystals, JANA and XP from the Shelxtl suite were some common choices of the day. James' tool of choice was XP created by the titan of crystallography, George Sheldrick. He explained to me a seemingly endless list of commands TELP, PROJ, FMOL, UNIQ, HADD, etc. as he refined the structure (P.S. some work in Olex2). I worried about how to remember all of these hand typed commands and scribbled some notes. When I looked back at the screen, a 3D picture of an iron complex was rotating on screen. We weren't finished yet, but in a few minutes it was pretty clear what the sample was. Here there was no inferred structure from an assembly of evidentiary techniques like NMR, MS, UV-vis and CHN. No. Here was a definitive picture of the molecular structure of the compound. I felt like I had discovered the cheat code. After spending my undergraduate years learning all about the 'standard' techniques and how important they were, this blew them out of the water. I was hooked.
A recent conversation reminded me of this first toe being dipped in the pool of crystallography and I thought I would put down some words about the changes I've seen in the technology over my 21 years in crystallography, (since that officially makes me a 'grown-up' crystallographer).
At the time of my first lesson, diffractometers were vastly different machines in performance terms, even if they didn't look so different. Area detectors were the norm, but based on CCD technology. The CCD detector we had then took about 4-5 seconds to read out an image making it far slower than the last generation of CCDs that were sold by diffractometer vendors and far, far slower than the HPCs we have today. This glacial readout time made molybdenum sources better suited to CCDs since fewer images meant fewer readouts. Microfocus technology was just around the corner and fine focus 3kW tubes with a graphite monochromator were the workhorse in most labs.
In these days, a 'fast' data collection was one you finished within the working day. Established crystallographers of the day may even have thought "How can diffractometers possibly get better than this?" having not long left behind a world of week long data collections with serial point detectors (which had to collect thousands of reflections one-by-one). An area detector, even a slow one, collected many reflections in parallel and dramatically cut experiment times for them. I also heard tales of refining structures on mainframes (e.g. the IBM 370 with punch card input) from older members of the crystallography community. I felt we already had it pretty good with PCs running the colourful Fisher Price themed Windows XP and capable of refining most structures in a few seconds.
So, how did it get better from my starting point? (Disclaimer: At this stage I have to remind you this article is written from my own personal perspective, reflecting the instrumentation I have personally used.)
At Edinburgh, our practice for the 5 years or so after 2004 was to operate with minimal downtime. Collecting data 7 days a week, one sample in office hours and one overnight meant we could stay on top of the steady stream of samples being submitted for analysis. Then, we used hand made mounts comprised of glass wool glued to the tip of a glass capillary. This was done to put less glass in the beam and therefore reduce diffraction from it. These were fragile beasts and could be seen oscillating in the flow of the cryo plume if the piece of glass wool was too long. During one of my weekend shifts, I planned to pop into the lab for an hour, mount a sample (like I had done many times before) and go off to enjoy the glorious Edinburgh weather. The crystallography gods were not smiling on me that day. Eight hours later and the air in the lab was now blue but an experiment was running. The glass bin had been topped up more than it should have with broken capillaries representing hours of remaking mounts, breaking them and starting again. The issue was exacerbated by the need to wait 20-30 minutes just for a unit cell, to find out if the sample was worth collecting.
Such instruments typically demanded large, say 300 micron or so, crystals that looked like gems for success to be within reach. Our departmental service users were often turned away in frustration if their samples didn't visually fit the bill, because we simply didn't have the time to test everything at the diffractometer.
In January 2010, I saw my first jump in performance. I had taken the staff crystallography position at University of Edinburgh School of Chemistry a year prior and settled into my grove when a new instrument arrived in our lab. The SuperNova. A man named Zoltan Gal, who I would come to know very well a couple of years later, stood at the controls, and started the first experiment of our training. Simon Parsons, Stephen Moggach, myself and a (what's the collective term for crystallographers?...) disorder of budding crystallographers huddled around the monitor. Very quickly, the first diffraction image appeared. Simon who was standing next to Zoltán turned around to the rest of us with a huge grin on his face.
So where did this performance jump come from? Microfocus sources operating at only 10s of watts use focusing optics to outperform the traditional sealed tube by making better use of the photons generated by the tube. Copper diffracts more strongly than molybdenum benefiting not only the diffraction from the crystal, but also the efficiency of the optics. The larger detector was spending more time collecting data and less time reading out.
With this instrument, data collections moved into the minutes/hours regime from the hours/days regime. It was possible to do more, do it better and do it faster. Concurrent data reduction was also a revelation. Getting a first look at the data automatically and after only a fraction of the data collection was complete saved wasted instrument time. As the performance improved, the standards for samples our service accepted naturally dropped. In hindsight, I don't think I fully exploited the performance of that particular SuperNova as best I could, but even so, our sample queue began to disappear and weekend working stopped being necessary.
In February 2011 only a year later, I started a new position with Agilent Technologies as an application scientist. I learned fast. Getting the inside track from the R&D team hugely benefited me as a crystallographer. I learned that I had been staying in my comfort zone and not pushing limits. I learned some behind the scenes information such as why Damian Kucharczyk had made design choices he did and how they resulted in phenomenal machines like the SuperNova. I learned that the microfocus sources could accommodate much smaller samples than our older instrument had conditioned me to consider acceptable or even that logic suggested. I now had the time to play, to push the limits to try to break things. I also was now using robust nylon loops and kapton mounts instead of glass which made sample mounting that much easier and had the added benefit of reducing background. Cutting-edge tools and a motivation to learn how to push their limits? It was a great period.
We had a customer come in for a demo who, sensibly, decided to bring more samples than he thought he'd need. By the end of the two days, we had collected 14 datasets and run out of samples. Some with molybdenum, some with copper, most with both. One of them we even collected in only 3 minutes after Zoltán spent a little time tuning the strategy. One of them was even deliberately smothered in glue to make a challenge with the instruction "get the best result you can in 1 hour". In Edinburgh we'd been conservatively collecting 2-3 samples per day. On my first demo, we were sitting at double that tally, with essentially the same instrument.
Over the next few years I seriously enjoyed demos. Our source performance doubled. Our detector performance doubled. The SuperNova goniometer began struggling to keep up with them (but a coming change would soon fix that). Software steadily improved. One of the main things I loved about demoing our instruments was the look on the faces of our guests when AutoChem (shout out to Horst Puschmann and Oleg Dolomanov) revealed their solved structure on the screen just a couple of minutes into the demo. Or logging in to the system remotely from my phone to show the customer that the structure had solved while we were out at lunch. We had to actively insist customers bring the worst samples they could find (we still do) because we were partly worried about running out of samples too early and partly because if it isn't difficult, it doesn't highlight advantages.
I remember Oliver Presly commissioned a video in around 2013 where a data collection was filmed using our new Atlas S2 CCD detector to show how fast CCDs and the workflow had become. Alexandra Stanley stood at the diffractometer like a statue for hours as the film crew captured every conceivable angle of the diffractometer and screen from over her shoulder. (Not sure how she kept her cool that day.) When watching back the first edit, we saw the "shutter closed" light flicker for the briefest moment periodically as the experiment progressed. This was one of several times in my career I paused to reflect on how far things had come. Several seconds of readout time between images was now down to about a fifth of a second. "How can diffractometers possibly get better than this?" I thought. At this time we were collecting data in about an hour for the typical sample that come through the demo lab, but already a few minutes was becoming more common depending on sample size, composition and symmetry.
In 2014 or so Mathias Meyer and I had a conversation during which at least part of the idea for "What is this?" (WIT) was born. I remember suggesting that we take an X-ray image every time we took a visual image during the sample movie. Mathias, as he usually does, already had a plan for a long time before we had this conversation and was waiting for the right technology. Per Mathias's recollection: "Dr. Rosario Scopelliti – told me (in 1998), if the sample is lousy, make the experiment such to collect 1Ang to I/sig 4 and if you can’t crack it like this, more resolution or intensity won’t help you."
So "What Is This?" was born with the concept: if our goal is just connectivity, how fast can we get it? What is the minimum we can do for reliable connectivity? CrysAlisPro, was already in good shape for this thanks to the automation workflows and concurrent data reduction approach that we already had at the time. After Mathias, Przemek Stec and team implemented it, we had an answer. We could do it in under a minute with our S2 CCDs, which we all thought was phenomenal. Even for difficult samples, 5 minutes spent on a WIT experiment could save you hours of data collection on something uninteresting, as it did for me during a memorable demo. Five minutes of WIT and the customer recognised that the sample was not what it was supposed to be, and was not needed. The strategy for a full data collection was predicting 15 hours. Five minutes to save 15 hours? Seems like a good trade off to me.
Once again I foolishly thought, "How can diffractometers possibly get better than this?"
Something happened in 2015. Rigaku Oxford Diffraction (ROD) was born after Rigaku acquired the single-crystal business from Agilent. With this, HPCs became available to us as Rigaku had been pairing them with their product line for a few years already. The new horizon was shutterless data collection and hardware X-ray photon counting. Although we had reduced readout time to a mere 0.22 seconds with our last generation of CCDs, it was still 0.22 seconds that could be better used collecting data. With an HPC in the mix, WIT now took 31 seconds. (Sidenote: that one extra second really bothered me and I spent an in inordinate amount of time trying to get it down to 30 seconds - all I needed to do was wait for faster CPUs and it came for free!).
That was 10 years ago.
Shortly after was the last time I thought (or allowed myself to think) "How can it possibly get better than this?" and the reason why was that I had naively allowed myself believe that the final incarnation of the SuperNova was already the ultimate diffractometer. I think some of our passionate customers agreed with me at the time. The installation of our first XtaLAB Synergy-S in the UK demo lab quickly proved me wrong. There was a factor of 4 performance improvement I was not expecting but that was borne out through demo after demo. Despite this, I will always have a soft spot for the SuperNova as it represented my first experience of a step change.
By 2025, I have been fortunate enough to be around for several step changes in our ROD product line. The introduction of microfocus sources, several generations of improved X-ray optics from our colleages at Rigaku Innovative Technologies, faster CCDs, HPC detectors, curved detectors, faster goniometers, access to rotating anodes and last, but by no means least, electron diffraction. I work with some very talented people and they always find a way to raise the bar and set a new standard.
So now that is all said, let's revise my title to a more appropriate question posed to you, the reader: "How will diffractometers get better than this?"
(This article was published originally on LinkedIn.)