AI Activation Functions
Keep nonlinear calculations numerically stable.
How it works
Activation functions let neural layers represent nonlinear mappings. Sigmoid maps a scalar to zero through one; softmax normalises several scores. Neither operation guarantees calibrated probabilities. Large exponents can overflow, so a stable sigmoid branches by sign and softmax subtracts the largest score before exponentiation.
Run the code
Try in playground loads this complete standard-library Python example without executing it. Choose Run when ready. Local review uses Python 3.12.14; the existing browser editor uses Pyodide 0.27.7. These small fictional examples need no account, external data, model-provider key or database service.
Example
import math
def sigmoid(x):
if x >= 0:
return 1/(1+math.exp(-x))
exp_x = math.exp(x)
return exp_x/(1+exp_x)
scores = [1000.0,1001.0]
shifted = [math.exp(x-max(scores)) for x in scores]
probabilities = [x/sum(shifted) for x in shifted]
print("sigmoid:", [round(sigmoid(x),3) for x in [-1000,0,1000]])
print("softmax:", [round(p,3) for p in probabilities])
Expected result
sigmoid: [0.0, 0.5, 1.0] softmax: [0.269, 0.731]
Subtracting the same constant from every softmax input preserves its mathematical ratios while keeping exponentials manageable. Real numerical work also needs finite-input checks. Saturated activations can have tiny gradients even when the forward calculation is finite.