Precision Examples in Math

Start with the recap, study the fully worked examples, then use the practice problems to check your understanding of Precision.

This page combines explanation, solved examples, and follow-up practice so you can move from recognition to confident problem-solving in Math.

Concept Recap

The degree of exactness in a measurement or calculation, reflected in the number of significant digits reported.

How many decimal places matter? Measuring in inches vs. millimeters.

Read the full concept explanation →

How to Use These Examples

  • Read the first worked example with the solution open so the structure is clear.
  • Try the practice problems before revealing each solution.
  • Use the related concepts and background knowledge badges if you feel stuck.

What to Focus On

Core idea: More precision means more decimal places and finer distinctions, but precision must match the measuring tool's capability.

Common stuck point: More precision isn't always better—it must match the context.

Sense of Study hint: Ask yourself: what is the smallest unit my measuring tool can detect? Your answer should not have more decimal places than that.

Worked Examples

Example 1

medium
A student measures a pencil three times and gets 15.2 cm, 15.1 cm, 15.3 cm. Another instrument gives 15.18 cm, 15.20 cm, 15.19 cm. Which set of measurements is more precise? Which is more accurate if the true length is 15.5 cm?

Solution

  1. 1
    Precision concerns spread (consistency): Set 1 ranges from 15.1–15.3 cm (range = 0.2 cm). Set 2 ranges from 15.18–15.20 cm (range = 0.02 cm). Set 2 is more precise.
  2. 2
    Accuracy concerns closeness to the true value (15.5 cm): Mean of Set 1 = 15.2 cm; mean of Set 2 = 15.19 cm. Both are far from 15.5 cm, but Set 1 is slightly closer on average.
  3. 3
    Conclusion: Set 2 is more precise (tightly clustered) but neither set is very accurate.

Answer

Set 2 is more precise; neither set is highly accurate relative to the true length of 15.5 cm.
Precision measures repeatability (how close repeated measurements are to each other), while accuracy measures correctness (how close measurements are to the true value). A measurement can be precise but inaccurate (systematic error) or accurate but imprecise (random error).

Example 2

hard
Add 12.3 cm and 4.56 cm, applying the rule for precision in addition.

Practice Problems

Try these problems on your own first, then open the solution to compare your method.

Example 1

easy
Which measurement is more precise: 7 m, 7.0 m, or 7.00 m? How many significant figures does each have?

Example 2

medium
Multiply 3.4 cm \times 2.15 cm, applying the significant figures rule for multiplication.
← Back to Precision Practice Mode

Background Knowledge

These ideas may be useful before you work through the harder examples.

decimal representation