Safety Tips: December 2024 Early Results for Backers

Note: This was an early release of data to backers with analysis based upon a superseded dataset. For the most up-to-date details on Safety Tips read Safety Tips Phase 1 Project Results

A month and a half ago we completed the experiments for Phase 1 of the “Safety Tips” research, trying to establish how different tipping methods or swords affect two different risks: the possibility of pushing on a mask potentially leading to concussion, and the risk of puncturing soft targets. Since then we’ve been working on analysing the data which turned up a few surprise issue. We promised early results to all backers before the end of the year and despite some challenges we have some results to share! This is not the final analysis, but we are confident in them with a few caveats that will be explained below. This article will give more detail on the tests and challenges, as well as initial results we have.

What experiments did we run?

In the run up to the experiments we went out to backers to understand their priorities of what we should be testing. This lead to deciding on different “experiments”, where each experiment is a group of conditions that we want to compare to both of the risks under investigation. (“Concussion Risk” and “Puncture Risk” for short). Each risk has it’s own test, so an experiment consists of running both tests for all of the conditions in the experiment.

As a reminder of the tests we ran for each experiment:

Concussion Risk Test: We placed a mask on a moveable boxing ball stand, with an accelerometer placed on both the sword and the target. Each experimenter struck the target in a defined pattern of strikes that specified the angle the mask was struck from and the part of the mask to aim for. This resulted in minimum of 108 strikes per condition. We recorded the acceleration on the target and the sword.

As an additional data point an experimenter stood to the side of the target and classed each strike into “skip”, “light” and “heavy” based upon how far the target moved in relation to the base. “Skip” meant the arm of the target didn’t move, “Heavy” meant that it moved beyond the edge of the base (30cm+), and light was any movement less than heavy. This was added as a back-up measure due to possible issues found with the accelerometer data in the pre-tests that we made changes to adjust for but couldn’t fully test ahead of the experiment.

Puncture Risk Test: We mounted the sword in a vertical drop rig that can be adjusted to drop from different heights. We then placed a target beneath it made up 3 layers (bottom to top): 8cm deep of 20% ballistic gel, 4mm close cell polyurethane foam, 1mm chamois leather.

The target medium was different to the original planned 10% ballistic gel only. The change in test medium is based upon materials used for testing less-lethal blunt rubber bullets in lab conditions – the switch was prompted by issues in the pre-tests with untipped swords too easily getting through the original targets even when simply rested on the target. The new medium simulates a human target better than ballistic gel alone, with the gelatine being approximate to muscle tissue, the foam approximate to a fat layer, and the leather approximate to skin. In some cases where tips were not able to puncture the above target we also tested against 20% ballistic gel with no additional layers (we refer to these as “skinless” targets).

We first used a “pilot” target to find an approximate height that would puncture the target. This same target was struck multiple times until it punctured – because this might be affected by repeat stresses, we don’t record this result as data. Then we drop the sword on several other targets at heights below and above the pilot puncture height. These all only have one drop, and we recorded if and how deep the target was punctured.

Between various challenges in the run up to the day and on it, we only had time to perform two experiments. These were:

Rapier Tips: A single cup hilt rapier (1094g weight, 7.6kg static flex) with Rounded Tip . We compared Untipped, Rubber, Bullet Casing and Thermoplastic

Longsword Brands: A comparison of 4 untipped longswords of different makes and models (Aureus Alexander III, Sigi Light, Sigi Maestro and Regenyei Strong)

Data Challenges

We have not yet completed all of our data analysis due to challenges with working with the accelerometer data having set back our timelines.

The short version is that the accelerometers did not like the jolt of impact when the target is initially struck. This creates a non-random error where it systematically under reports acceleration, and the harder the hit the worse the under-reporting – very bad.

We had identified this risk before the experiment and made changes in line with advice from the manufacturer on how to compensate, but they didn’t end up solving the issue. Investigating and trying to find solutions for this issue has eaten up most of the time we had allocated for the first round of data analysis.

We’re working on a solution using the angle data on the sensor to calculate how far the target moved, which should allow us to get a more detailed analysis. This works because the sensor is in a fixed position and will move in an arc, so angle is all we need to calculate distance. Initial tests show this has promise but we’re still working through the data to make sure our assumptions are valid. The visual check that we added as a back up plan is a less robust version of this same test – and the results below are using the visual check instead of the accelerometer version.

We had a different problem with the Puncture Risk Test: the differences in tip types for rapier were so extreme that we couldn’t puncture the new test medium with some of the tip types, and even in some cases just with ballistic gel uncovered by the other layers. We can say we’re pretty confident this result is significant! But it does make the statistics to show that slightly more complicated (at least for the rapier tips experiment).

By contrast, the Longsword Brands all punctured the normal test target but the differences are very slight. The switch to 20% gel also halved the number of targets we could make, making it hard to find significant results when differences as so small. Most of our focus is on getting the acceleration data working, so we have done some basic statistical tests but need to consider some other options to account for these data issues.

Results

You probably just skipped to this point without reading anything above so just in case you did: these are initial results with caveats based upon the challenges described above.

Rapier Tips Experiment

Concussion Risk Test Results

The Concussion Risk Test found a clear difference between the different tip types.

In total we recorded 459 strikes across all conditions (108 per condition, and 27 extra untipped strikes as a spreadsheet error duplicated some of the strikes for one experimenter before we noticed).

We ran a Fisher’s Exact test to test for significance on the observed data, comparing each tip type to the behaviour of an untipped rapier. We used Fisher’s Exact over Chi-Squared as some of the categories were close to the low-end of accuracy for the Chi-Squared test.

At the extreme end of the spectrum, rubber tips were 2.4x as likely to catch on the mask as an untipped sword (p-value: 0.00002, highly significant). Bullets were also more likely to catch, although to a lesser extent at only 1.5x as likely (p-value: 0.02). The difference between the thermoplastic tip and the untipped rapier were not significant (p-value: 0.73).

The heavy hits showed a similar trend, although we would like to do more analysis using the better version of the data before discussing that in detail as it looks like the peak-force may also have interesting differences that need to be explored.

Unfortunately at this stage we aren’t able to match the target results with the level of force in the sword strike: that will come when we have resolved the issues in data analysis described above.

Puncture Risk Result

This test also resulted in clear patterns. As described above the differences were far more extreme than we expected. The untipped rapier almost immediately started puncturing even our improved target at a drop height of 2cm (~1J Energy). The bullet required nearly 12x the energy to puncture, and still punctured to about half the depth of the untipped rapier at 1J. Unsurprisingly an ANCOVA test confirmed that this relationship was significant (p-value=0.000008).

The big “problem” is that both the rubber and the plastic were so much better than these tip types that we couldn’t puncture the target at a height we were willing to drop the sword at. To give you an idea, here’s a video of the plastic tip failing to puncture a lower resistance 20% ballistic gel target with no additional layers at a a 50cm drop (30J)

We didn’t want to go higher than this for fear of damaging the sword. Judging by the flex on the blade this drop energy seems to be comfortably within normal ranges for a strike. Again, this target is weaker than the one used for the previous graph as it lacks the foam and chamois layers.

For comparison, both bullet and untipped rapiers will puncture this target simply by being placed on top of it with the weight of the drop rig, although very slowly in the case of the bullet. At 5J, the bullet case punctures twice as far (4cm) in this medium as the highest depths we recorded at 12J for a bullet on the other target.

Yet at 2.5x the energy required for a bullet case to puncture the stronger target, a plastic tip is still failing to puncture this weaker target.

The surprise result was that plastic actually out performed rubber. Rubber also failed to puncture the target with a “skin” layer but did start puncturing the “skinless” target at around 14J, although it did not reliably puncture across the tested ranges and sometimes failed to puncture at higher energies. Apart from material, the main difference between rubber and plastic is the shape – the rubber tip is concave and the plastic tip convex.

The last interesting thing to note is that the differences we found outstrip what you’d expect based upon surface area alone. The bullet tip is 4.5x the area of the untipped rapier, but needed 12x the energy to puncture the target. The rubber and plastic tips are the same 17mm diameter, which results in only 2x the area of the bullet tips but a much bigger increase in energy to puncture.

It is not clear at this stage how shape, material and area are interplaying to produce these results, but it is clearly a combination of factors.

Longsword Brands Experiment

So what about comparing the longswords? Well, the most interesting thing we can say about the longword brand comparison is that the results were much less interesting and pronounced for both tests.

As you can see, all four of these are closer to each other than any of the rapier tips were to another type of tip. We did the same tests as above, but as there was no “baseline” sword instead we compared each sword to all other swords.

The only significant result we found was that the Aureus was more likely to result in a hard strike than the others (p-value=0.0025) although the differences in how often it skipped were not significant. We found a couple other results to be not quite significant with the above data and expect that when we do more detailed analysis with the richer dataset we might find some other differences as well as it will allow us to be more exact on understanding the force in the strikes.

The puncture test was even more inconclusive. Using the same ANCOVA test that we used above, we didn’t find any significant differences here. This is partly because of the low number of targets we had due to needing to increase to 20% ballistic gel, so we’re still exploring if another test is more appropriate for the data.

One thing worth noting however is that all of these swords punctured the target at 4-7.5J- quite a bit below the 12J from a rapier with bullet tip, but still well over the 1J from an untipped rapier. This certainly suggests that different tip types on the sword are having the desired effect compared to rounded tips.

The lack of clear results are not hugely surprising as the differences between each model of longsword are quite complicated. Each has a different tip shape, area and flexibility from the others: we have no idea how much each factor affects these risks and it is likely that they are each interplaying with each other to create these results.

Conclusions & Next Steps

Our initial conclusion is pretty simple: untipped rapiers are very dangerous, and while rubber tips drastically reduced the chance of puncturing the target, they were very noticeably more likely to push on a mask. The thermoplastic tips were the obvious winner. They were the best performing tip for reducing the risk of puncturing and not significantly worse than an untipped sword for pushing on a mask. While further analysis will likely help us understand this result in more detail, this broad conclusion is unlikely to change.

As for longswords: quite simply more research is required. Any differences we do find over and above those discussed will be slight and likely caused by a wide range of contributing factors. Unpicking how things like flexibility, tip area and shape affect risks independently and in combination will require several other tests.

Obviously, we still aim to publish more detailed analysis in the new year based upon the full accelerometer data, as well as a video showing the results and the experiment in more detail. This will likely be first published through the Historical Fencing Research website as “Phase 1” rather than an academic submission – although we will go through the same analysis process as we would if submitting directly to a journal. This will allow people to make decisions based upon these results while we still investigate other important questions.

Once phase 1 is completed, we will be looking at creating a pipeline of tests and understanding how to fund them. We’ll combine the phase 1 results and the additional tests for a journal submission that has looked at a broad spectrum of factors. Rest assured, however or wherever we publish we’ll make sure the results are open access.

All of the equipment we’ve built is still constructed and ready to go. We believe we can adapt the target to resolve the accelerometer issues now that we better understand what is causing them, and there is clear value in performing the same tip tests on other weapons, as well as exploring how material, shape, diameter and/or flex are affecting the results.

There was also high interest from backers for adding protective equipment into the equation: we agree this is an important angle of study. Doing that properly requires some adjustments to these test methods, and we need to consider other testing apparatus to also test protection against cuts – this is still in the planning phase. Likely this will be a separate line of research to the main Safety Tips, although much of the methods and equipment will be the same.

In short, we hope this is just the beginning for Safety Tips research – stay tuned.

What can you do and say?

First of all, a huge thank you to all of you who made it possible to get this far – these experiments have taken up a lot of time and required some fairly expensive purchases and we couldn’t have done it without you.

Obviously, the results above are provided with plenty of caveats. Even so you may decide you want to act on them right away and discuss them with non-backers in your HEMA circles: that’s perfectly fine and entirely up to you.

If you’d like to discuss these findings with colleagues it’s OK to share the article with them, but please keep this to a small number – i.e. discussing with your tournament or club committee is fine, broadcasting it to the whole of your association is not. Please feel free to discuss the findings more widely if you wish, but if you do be sure to stress that these are initial findings and we may have more nuanced explanations later when the full analysis is complete – and let people know to follow our website and Facebook page for updates.

We would suggest the only thing worth even considering acting on is the tipping-methods experiment, as we really don’t yet have any firm conclusions on the longsword comparisons yet.

The good news is that if you want to change your sword tips, thermoplastic is cheap, easily available and simple to apply to any type of sword. In the UK at least the most readily available brand is called “Whitemorph”, but it comes under other names like “Thermoplast”, “Polymorph” or many other names.

This is already the recommended tipping method for rapier tournaments in the UK. If you want to start using it ahead of the full results, our project lead produced this video for UK tournament attendees demonstrating how to work with it. 10-15g of plastic is more than enough for the 17mm diameter tip we used in the test.