Rendered at 12:17:58 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
weinzierl 22 hours ago [-]
The "integer arithmetic paradigm" would mean either checking after (almost) every operation or living with potentially incorrect results.
The first is terribly inefficient, the second is just wrong (even if insanely common).
This is sad because there is no reason at all why integers couldn’t follow what OP calls the "float paradigm". It doesn’t even have to be slow. Most modern processors (if we ignore x86) support some form of sticky arithmetic flags. Unfortunately, programming languages don’t support them, so they aren’t used.
There is also something to be said about
the special treatment division by zero gets.
Someone 5 hours ago [-]
> The "integer arithmetic paradigm" would mean either checking after (almost) every operation or living with potentially incorrect results.
Would an int type with three special values +inf, -inf and NaN be useful? In addition to handling overflow, it would be symmetrical; the relation
>I sometimes wish that signed integers were symmetrical. i8 would represent the range of [-127 to 127] with 0xFF representing NaN. Any operation which can not be computed (division by zero, overflows, operation with another NaN, etc.) would result in NaN. For further symmetry we could do the same for signed integers as well.
ErroneousBosh 2 hours ago [-]
> i8 would represent the range of [-127 to 127] with 0xFF representing NaN
There's an interesting "special case" of that in the Alesis HR16 and SR16 drum machines. They store 16-bit sample data in an 8-bit ROM. Okay, you say, nothing unusual about that, it's int16 pairs, right?
Nope.
It's int8 data, that gets scaled. Drum sounds tend to decay, so you start off with the 8 bits of sample data representing the eight highest bits of the 16 bit DAC value. At some point the maximum level has decayed by a half so all subsequent values will fit in 7 bits. So there's a "tick" to -128 in the data that makes the sound ASIC "change up a gear" and shift the 8-bit data right one place with the data scaled to suit. Once it decays enough again, another gearchange, another scaling, another "notch" on the Christmas tree-shaped sample.
At the very end of the sample to signify that the note is to stop, you just have a tail of -128s until the shifter has "run off the end" - from memory (it was around 2010 that I last looked at this) there are eight or nine "ticks" in total which are necessary for the sample to stop before it goes right on into the start of the following sample.
So, yeah, nothing new under the sun. Signed 8-bit audio with -128 considered a special value.
dnautics 21 hours ago [-]
Pretty sure zig has safe integer operations: integers aren't allowed to overflow with the standard operators, there are overflowing and saturating operators if that's what you want.
weinzierl 15 hours ago [-]
Rust and D and probably others have this too. The interesting question is:
Can you chain these operations in a way that will propagate the error to the end result while simultaneously have a compiler produce code without branches after every operation. Safe integer operations alone are not enough for that.
dnautics 14 hours ago [-]
I think the operators panic. In zig, If you want safe operations I think there are functions in the stdlib
The branching does not cost as much as you think, I'm pretty sure the compiler marks the erroring path as cold, which the cpu can use to elide most of the branch cost through specex
weinzierl 14 hours ago [-]
A branch is a branch and no branch will always be better. With sticky flags there is no difference in performance to regular integer arithmetic.
kibwen 20 hours ago [-]
> Most modern processors (if we ignore x86) support some form of sticky arithmetic flags. Unfortunately, programming languages don’t support them, so they aren’t used.
I'd say the reason that programming languages don't have built-in support for checking the CPU's sticky status flags is precisely because of a lack of x86 support. Plenty of languages do have support for checking overflow on each individual operation, e.g. Rust's `overflowing_foo` methods, which return a tuple whose second member is a boolean indicating overflow: https://doc.rust-lang.org/std/primitive.i32.html#method.over...
jcranmer 20 hours ago [-]
Sticky flags tend to be really annoying for a compiler to support for various reasons, but principally it makes every operation have a hidden dependency to shared global state that is almost never read. Note that IEEE 754 standardized support for sticky flags 40 years ago, but support for these sticky bits is still poor to nonexistent in most programming languages, and sticky bit stuff for floating-point operations is less problematic than integer operations because FP math is already far more black box in practice.
a_e_k 19 hours ago [-]
For the float stuff, beware of `-ffinite-math-only`. If enabled, `isfinite()` and may compile out as assumed true (with similar assumptions around `isnan()` and `isinf()`).
And `-ffinite-math-only` is enabled by `-ffast-math` which in turn is enabled by `-Ofast`.
toolslive 19 hours ago [-]
Not only that: `-ffast-math` might change the result of flops when the arguments are regular floats too. (It abandons IEEE754 compliance)
Maxatar 14 hours ago [-]
I was skeptical about this article's claim about LLMs so I tried it out myself. Looks like there are several layers of pedantry involved, for example, strictly speaking in base C it's undefined behavior to divide by 0.0, even for floats, so the author's initial implementation is incorrect and a optimizer could elide the call to isfinite in certain scenarios where the denominator is 0.0.
Presumably the airtight implementation assuming Annex F is as follows:
#pragma STDC FENV_ACCESS ON
int my_div(float x, float y, float* r) {
if(!r) {
return -1;
}
fenv_t environment;
if(feholdexcept(&environment) != 0) {
return -1;
}
float result = x / y;
int failed = fetestexcept(
FE_INVALID | FE_DIVBYZERO | FE_OVERFLOW | FE_UNDERFLOW);
if(fesetenv(&environment) != 0 || failed != 0) {
return -1;
}
*r = result;
return 0;
}
When I asked about the author's implementation I got the response that the author's implementation is deficient in several areas:
- Underflow produces a finite result and passes this check.
- Overflow can produce a finite maximum value under some rounding modes.
- Quiet NaNs and valid infinite results fail this check, even without an arithmetic exception.
It also assumes floating-point traps are disabled. Annex F defines the arithmetic behavior; your function still needs a chosen definition of “failure.”
So the answer really is... safely dividing two floats really depends on what trade-offs you're willing to make, and the original reply of just checking for a 0 denominator is honestly the most sensible, portable, and safest.
dooglius 22 hours ago [-]
FWIW one can configure floating point exceptions at runtime at no overhead, see `man 3 fenv`
jcranmer 20 hours ago [-]
It's no overhead in the same sense that zero-cost exception-handling is zero-cost: turning them on properly (e.g., #pragma STDC FENV_ACCESS ON or equivalent command-line flags) disables a fair amount of optimizations which incurs an overhead in and of itself.
dooglius 17 hours ago [-]
Oh interesting I thought it was more or less a wrapper around MXCSR (or equivalent for other architectures)
jcranmer 13 hours ago [-]
That is what you have to do enable turning FP exceptions into traps on a hardware level. But the FP instructions your compiler actually emits as hardware instructions bears only a superficial resemblance to the code you originally wrote. Compilers generally assume FP operations are pure operations without side-effects, so it happily speculates them (including ones that might generate the trap you're looking for) on paths where they didn't, or remove them, or evaluate them at compile time instead of run time, etc. Turning this behavior off so that the traps you get are actually the traps you expected to get involves disabling a lot of optimizations.
dzaima 20 hours ago [-]
Except gcc doesn't actually support it to any sane extent (it ignores and warns on "#pragma STDC FENV_ACCESS ON", which C requires for fenv.h to actually function; and as such gcc makes a bunch invalid optimizations); clang supports it, but only as of somewhat-recently.
The first is terribly inefficient, the second is just wrong (even if insanely common).
This is sad because there is no reason at all why integers couldn’t follow what OP calls the "float paradigm". It doesn’t even have to be slow. Most modern processors (if we ignore x86) support some form of sticky arithmetic flags. Unfortunately, programming languages don’t support them, so they aren’t used.
There is also something to be said about the special treatment division by zero gets.
Would an int type with three special values +inf, -inf and NaN be useful? In addition to handling overflow, it would be symmetrical; the relation
would hold.>I sometimes wish that signed integers were symmetrical. i8 would represent the range of [-127 to 127] with 0xFF representing NaN. Any operation which can not be computed (division by zero, overflows, operation with another NaN, etc.) would result in NaN. For further symmetry we could do the same for signed integers as well.
There's an interesting "special case" of that in the Alesis HR16 and SR16 drum machines. They store 16-bit sample data in an 8-bit ROM. Okay, you say, nothing unusual about that, it's int16 pairs, right?
Nope.
It's int8 data, that gets scaled. Drum sounds tend to decay, so you start off with the 8 bits of sample data representing the eight highest bits of the 16 bit DAC value. At some point the maximum level has decayed by a half so all subsequent values will fit in 7 bits. So there's a "tick" to -128 in the data that makes the sound ASIC "change up a gear" and shift the 8-bit data right one place with the data scaled to suit. Once it decays enough again, another gearchange, another scaling, another "notch" on the Christmas tree-shaped sample.
At the very end of the sample to signify that the note is to stop, you just have a tail of -128s until the shifter has "run off the end" - from memory (it was around 2010 that I last looked at this) there are eight or nine "ticks" in total which are necessary for the sample to stop before it goes right on into the start of the following sample.
So, yeah, nothing new under the sun. Signed 8-bit audio with -128 considered a special value.
Can you chain these operations in a way that will propagate the error to the end result while simultaneously have a compiler produce code without branches after every operation. Safe integer operations alone are not enough for that.
The branching does not cost as much as you think, I'm pretty sure the compiler marks the erroring path as cold, which the cpu can use to elide most of the branch cost through specex
I'd say the reason that programming languages don't have built-in support for checking the CPU's sticky status flags is precisely because of a lack of x86 support. Plenty of languages do have support for checking overflow on each individual operation, e.g. Rust's `overflowing_foo` methods, which return a tuple whose second member is a boolean indicating overflow: https://doc.rust-lang.org/std/primitive.i32.html#method.over...
And `-ffinite-math-only` is enabled by `-ffast-math` which in turn is enabled by `-Ofast`.
Presumably the airtight implementation assuming Annex F is as follows:
When I asked about the author's implementation I got the response that the author's implementation is deficient in several areas: So the answer really is... safely dividing two floats really depends on what trade-offs you're willing to make, and the original reply of just checking for a 0 denominator is honestly the most sensible, portable, and safest.> These eleven functions were defined in C99, and describe the handling of floating-point rounding and exceptions (overflow, zero-divide, etc.).