All Questions
Tagged with double floating-point
848 questions
2
votes
1
answer
179
views
How can I handle floating-point precision in C when dealing with numbers very close to zero or other edge cases
I wrote a program in C that calculates three points based on three points that are given. So we are given points A, B, C and calculate points A', B', C'. Now there are three ways that these points ...
- 31
0
votes
1
answer
173
views
Double multiplied with int has become less accurate in .NET Core
I was experimenting with this code block with different versions of .NET:
using System;
using System.Linq;
public class Program
{
public static void Main()
{
...
- 9,762
1
vote
1
answer
78
views
Precision issues when converting double to float
JDK has a random method in ThreadLocalRandom:
nextDouble(startInclusive, endExclusive)
But no float version. Here I want to float version, this is my implementation:
public static float nextFloat(...
- 2,089
-1
votes
2
answers
125
views
If Statement Cannot Evaluate Basic Floating-Point Problem
I was working through Bjarne Stroustrup's "Programming-Principles And Practice Using C++" book and I'm currently stumped by the 5th "Drill" question of Chapter 4.
There is a series ...
- 11
0
votes
1
answer
129
views
Rounding when ULP 1 - Java and C++
The reason of the question originated from differences in the results that I observed when computing std::exp(x) in C++ against Math.exp(x) in Java (with x of type double). I noticed that the ...
- 29
0
votes
1
answer
86
views
float/double is not working on qemu and spike for riscv-unknown-elf-gcc tool chain
float/double is not working on qemu for riscv-unknown-elf-gcc tool chain
it is giving me exception trap_illegal_instruction
I just tried running a simple program which has float in it, using riscv-...
0
votes
1
answer
68
views
How much accuracy is lost due to representing time as a IEEE 754 binary64 number?
My python program has a busy loop. Within the loop, some delays are calculated in the following fashion:
timeout_s: float = args.timeout_s
timeout_end_s: float = time.time() + timeout_s
while True:
...
- 140k
5
votes
1
answer
110
views
Determining the Upper Limit of Math.random() Output in Java
I'm seeking clarification on the upper limit of the numbers generated by the Math.random() method in Java. While I understand that the range of this method is "greater than or equal to 0.0 and ...
- 92
1
vote
3
answers
94
views
Why is double printing the same value as float in C when they have different sizes?
I need to do calculations using π. When I checked precision of M_PI, it only printed 7 decimal digits. So I #defined my own MY_PI. But I am still facing the same problem. double should support 15-17 ...
- 218
-2
votes
1
answer
112
views
How Can Calculators Display So Many Digits of PI?
I was playing around with the calculator on my phone and I discovered I can see arbitrary numbers of digits of pi on there. I know double precision floats only have accuracy up to ~16 digits so I ...
7
votes
2
answers
216
views
strtod edge of denormals glitch
I have been looking at some code which had a weird optimisation fault and, in the process, stumbled upon a fault condition in strtod(), which has a different behaviour to strtof(), right on the edge ...
- 2,554
0
votes
3
answers
88
views
Cannot convert from double to float Error
I'm writing a simple program that converts pounds to kilos, but I'm a bit confused
import java.util.Scanner;
public class FiveTenOne {
public static void main(String[] args)
{
Scanner ...
0
votes
0
answers
226
views
Performance of float vs double with GCC, Intel & MS C
I have seen the previous ancient x87 era thread on this and thought it was time to revisit it in the modern era of SSE2 & AVX. The results of my fairly simple C benchmark test were mostly what I ...
- 2,554
1
vote
1
answer
59
views
How to unpack the TEDS ConRes type?
In a TEDS (Transducer Electronic Data Sheet) Template, there are two data types described as
ConRes.
"Constant resolution. This is a custom data type for compressed
floating point values that ...
- 5,841
1
vote
1
answer
195
views
Double precision floating point. Why is the maximum exponent 1023
When we have a binary Intel processor and a double precision float represented by 64 bits of which 1 bit for the sign, 52 bits for mantissa and 11 bits for the exponent.
I do not understand why the ...
- 290