New to Nutbox?

An Elusive Limit Question 6: Subtracting Nearly Identical Numbers can BREAK Calculators!

1 comment

5 days agoDTube2 min read

In this video I explain that the reason calculators gave an unclear limit for the complicated trigonometric function is because every term was nearly identical, to the point where subtracting them resulted in a difference too small for the precision of most calculators. The calculator would then have to round up or down, which can give the wrong answer. For example 1.0000000000123 - 1 = 0.0000000000123. If a calculator doesn't have more than 10 digits of precision, the difference would be rounded up to 0. And if this was in the numerator, as in the limit, then we obtain an indeterminate or undefined form of number / 0.

I also illustrate this by graphing the function and every term of it in Desmos graphing calculator. I zoom in beyond the precision of most calculators yet the terms still appear identical: https://www.desmos.com/calculator/vhwpqsail1

Time stamps:

Question 6: Explaining the unclear results of Questions 1 and 2: 0:00
Solution: Strange calculator results are due to numerator and denominator having very close and small values: 1:16
Zooming in the origin with Desmos graphing calculator: https://www.desmos.com/calculator/vhwpqsail1
All terms in the function are similar near x = 0: 2:29
All terms are near identical even when zoomed beyond precision of most calculators: 3:18
Example using 10 digits of precision: 3:25
- Same Example but calculating the difference first: 4:24
Dividing differences that are rounded to 0 is undefined: 5:01

Full video below: