What Is Calibration Tolerance?

Featured Product from Metal Cutting Corporation

What Is Calibration Tolerance?-Image

What Is Calibration Tolerance?
Specifying a Device’s Capabilities and How It Is Calibrated

Here at Metal Cutting Corporation, calibration tolerance is yet another important aspect of our Quality Management System (QMS). Whenever possible, calibration of the devices we use for measuring parts is based on NIST standards. For example, as we talked about in our recent blog “The Quandaries of Calibration Standards,” we use NIST-traceable pins to establish data points for calibrating our laser micrometers.

However, calibration tolerance is a variable thing, dependent on the device that is being calibrated and what is going to be measured with that device.

The Key to Calibration Tolerance
What do we mean by that? Let’s look at those laser micrometers we use at Metal Cutting. Each has a range in which it will work, and each device itself has its own tolerance. However, since we typically measure in a narrower range, we calibrate at the tighter range that we are working to rather than the broader range a machine is capable of measuring. For instance, a machine might be capable of measuring diameters from 0.010” to 0.250” (0.254–6.35 mm), but we are interested in a range of 0.020” to 0.100” (0.508–2.54 mm); therefore, we would use pins in that tighter range and use that library to set the calibration of the machine.

An interesting phenomenon of manufacturing is that the larger the diameter, the tougher it is to hold a tighter tolerance. So, for example, a Class XXX gage calibrated to three decimals for a 1.0” (25.4 mm) diameter would have a tolerance of ±0.000010” (0.000254 mm), but for a 10”.0 (254 mm) diameter the tolerance of the same Class XXX gage would be ±0.000050” (0.00127 mm). Typically, Metal Cutting works towards diameters of 0.004” to 0.067” (0.102–1.702 mm) while our machines are capable of measuring much larger diameters. However, since calibration tolerance also depends on what the machine is capable of, if a machine can only measure to ±0.000020” (0.000508 mm), obviously you must calibrate to within that range.

Therefore, the key to calibration tolerance is to make sure you specify both what the device is capable of doing and what it is calibrated to. That way, when you are using the device, you can be sure you’re not trying to measure something requiring a tolerance that is tighter than the device’s capabilities.

Certified by a Third Party
At Metal Cutting, we regularly send our NIST-traceable pins to an independent ISO certified lab for calibration, to ensure they are within specifications and to maximize their functionality. For instance, we send out our XXX pins to be calibrated within what an XXX gage normally measures at — that is, a calibration tolerance of ±0.000020″ (0.000508 mm).

ISO certified A2LA labs have equipment an order of magnitude better than what they are certifying. While Metal Cutting has very precise equipment in house, we don’t have the capabilities to calibrate to a level of, say, 0.0000001” (0.00000254 mm). Additionally, with that high level of precision, you also must account for the effects of thermal coefficient of expansion, requiring measurements to be done under the same tightly controlled temperature, humidity, air pressure, and other strict environmental conditions. Therefore, we rely on ISO certified third-party calibration houses for calibrating our pin gages; this in turn ensures our in-house devices are calibrated using tools that have the proper capabilities and tolerances.

While calibration tolerance in ISO 9001: 2015 is not specified, clause 7 of the latest ISO standards does make recommendations for choosing, maintaining, and calibrating traceable measuring equipment. (A side note: Although Metal Cutting Corporation itself is not an ISO certified calibration house, we have been approved for certification in the new ISO 9001:2015 standards for quality management and risk mitigation!)

Weighing in on Calibration Tolerance
While in our world calibration tolerance often means dimensions, it can also refer to weight. That is because, although Metal Cutting’s customers do not ask for parts of a discrete weight, our parts are so small and produced in such large quantities that we often count parts by weight rather than individually counting tens of thousands of incredibly tiny parts. Instead, we use the weight of 100 or 1,000 parts as a reference for an accurate counting scale. Therefore, we also routinely send our weight standards out to a certified lab for calibration, so that we can then use the weights internally to calibrate our scales for counting parts. Our in-house equipment includes multiple scales of different capacities and different tolerances, for our world of tiny parts ranging from grams to fractions of a milligram.

Calibration Tolerance vs. Process Tolerance
In addition to calibration tolerance, here at Metal Cutting we also take into consideration the tolerance to which we are cutting a part during the production process. The rule of thumb is calibration tolerance must always be tighter than the process tolerance, essentially applying the tried and true 10 to 1 rule. So, for example, if the tolerance we need to hold for our process is ±0.001″ (0.0254 mm), then we would use a laser micrometer with a calibration tolerance of ±0.00010” (0.00254 mm) — and the tolerance of the Class XXX pin used to calibrate the micrometer would need to have been even tighter!

Critical to Precision Manufacturing
Understanding equipment capabilities and tolerances is critical to precision manufacturing. Here at Metal Cutting Corporation, where we produce thousands of small metal parts every day, our QMS standards and our firm grasp of calibration tolerance help us ensure that we deliver high-quality parts that meet customer specifications.

Picking the ideal manufacturing partner for your needs will not only help you achieve the quality you want — it will also help to ensure that your production costs stay on budget and your deadlines are met.  For tips on finding the best partner for your metal fabrication needs, download our free guide, 7 Secrets to Choosing a New Contract Partner.

At Metal Cutting Corporation, we are precious metal cutting specialists, manufacturing burr-free tight tolerance parts from all metals. We provide the precision required by medical device, electronic, automotive, biotechnology, semiconductor, aerospace, fiber-optic, electrical and many other diverse industries that demand perfection:



Metal Cutting Corporation manufactures burr-free tight tolerance parts from all metals. We provide the precision required by medical device, automotive, electronic, biotechnology, semiconductor, aerospace, fiber-optic, electrical and many other diverse industries.

We are specialists with over 45 years cutting, grinding, lapping, polishing and machining metal parts. Our experience, inventory and capabilities provide the skills and capacity to meet the needs of technology device manufacturers. Specialty metals, micron tolerances, low or high volumes, complex metrology--all these and more are the requirements we achieve every day for products shipped worldwide.