Separate chaining Created 2026-10-13 Updated 2026-10-20
With separate chaining, each hash bucket in a hash table contains a collection of records, often a linked list. Lookup computes the bucket index, then follows that chain until a key equality comparison succeeds or the chain ends. A hash collision therefore affects the search work without merging unequal keys. Deletion searches the same chain and unlinks the matching record; in a singly linked list, remembering the preceding record during the search permits the unlink.[ref]
For the four-key integer example in hash table, insert a new record at the head of its chain after checking that its key is absent. Inserting in that order makes hash bucket hold , hash bucket hold , and the other two hash buckets empty. All five subsequent operations below select hash bucket because :
Separate chaining traces lookup, replacement, deletion and insertion through key equality
. Original deterministic model with . The table counts actual keys inspected, not elapsed machine time. Operations occur from top to bottom; only an absent key adds a new record, and the untouched record remains in hash bucket . Details.get(5) rejects key , accepts key , and returns after two key equality comparisons. get(13) rejects and reports absence after three comparisons. put(5,B*) examines and replaces the value without changing . delete(9) examines just the head record, removes it, and gives . Finally, put(13,E) rejects before inserting the new head record; returns to and the chain is . The successive hash table load factors are . These counts follow from the stipulated chain order; a different insertion policy changes the successful-search counts.For a chain of length , lookup and deletion by key take time complexity in Big O notation. The same applies to
put before any resizing, since replacement semantics require checking for an existing equal key. Head insertion alone is if absence is already known; that is a different precondition. These bounds assume bucket-index hashing, array access, key equality comparisons, allocation and link updates. A long string or large integer can make hashing or comparison depend on its encoded length, so counting record visits alone does not establish constant running time.Under simple uniform hashing, each distinct key's bucket index is an independent random variable with a uniform distribution on a finite set of indices. The keys and query are fixed independently of this random assignment. Then lookup has expected time complexity, where is the hash table load factor; it is expected when stays bounded. These are model assumptions, not an automatic consequence of using a deterministic hash function.[ref]
Lecture 8: Hashing I. 6.006 Introduction to Algorithms, Fall 2011 by Massachusetts Institute of Technology, 6.006 course staff; pp. 4–5, chaining worst case and simple uniform hashing assumption
Here is the short calculation. For an absent query and stored keys , let be the indicator random variable of . The number of records inspected has expected valueby linearity of expectation. For a present fixed query, scanning its entire chain inspects at most itself plus all other colliding records, with expected value ; stopping at the match only decreases this upper bound. Adding constant hashing and bucket access yields expected for either query. The argument needs pairwise collision bounds; it does not require all the indicator random variables to be independent. A universal hash family can provide a related bound by random choice of a hash function for a fixed key set, as explained in that article.
The worst case remains , and an absent query takes comparisons if all records occupy its hash bucket. For instance, all collide under . Separate chaining remains correct, but a sequence of insertions that each checks the existing chain can take quadratic total work. Amortized analysis of resizing does not remove this search cost, and an adaptive choice of keys based on a known hash function is outside the stated simple uniform hashing model.
Singular perturbation Created 2026-09-24 Updated 2026-10-14
A singular perturbation is a parameter-dependent problem whose limiting problem changes qualitatively as the parameter tends to zero. In an ordinary differential equation, a small coefficient of the highest derivative can reduce the order of the limiting differential equation, preventing it from satisfying all the original boundary conditions. Different length or time scales often require different asymptotic expansions.[ref]
Asymptotic Analysis and Singular Perturbation Theory by John K. Hunter. §1.1.2, p. 2: regular and singular perturbations
Consider the boundary value problemThis second-order differential equation is exactly solvable. Setting gives the first-order linear differential equation , so . One integration and the two boundary conditions giveBoth boundary conditions hold for every positive . For numerical evaluation, write the same ratio as , where evaluates the exponential function minus one without catastrophic cancellation near . Keep the denominator for finite : at it is approximately , rather than exactly one.
Setting gives . The outer solution selected by the right boundary condition is therefore , which fails the left boundary condition. The exact exponential function shows how that mismatch is repaired: the solution rises from zero to almost one within a boundary layer of width , using Big O notation. For each fixed , , while always. This pointwise convergence fails to be uniform convergence to the outer solution on , since for every .
Exact solutions with a shrinking left boundary layer
. The exact normalized profiles at , , and satisfy both boundary conditions. The dashed outer solution misses the left boundary condition. Colored dots mark , where most of the initial rise has already occurred. Details.The coefficient is small, but its product with the second derivative need not be small. Direct differentiation givesAt with fixed , these derivatives grow as and respectively as . Consequently and have exactly equal magnitudes and opposite signs. Even away from the boundary layer, their magnitudes remain equal in this example; both become exponentially small at each fixed positive . The outer expansion neglects an exponentially small transient there, rather than establishing that one derivative term is smaller than the other everywhere.
To predict the width without knowing the exact solution, suppose an order-one change occurs over distance . The derivative scales are and . The dominant balance gives . The resulting inner variable makes the boundary layer visible at a fixed scale. This is the useful feature of singular perturbation: an apparently disappearing term identifies the missing structure when one measures its derivatives on the appropriate scale.
Small rotation commutator 2026-10-14
Use the cross-product Lie algebra convention , , , so . Put and , using the matrix exponential. The finite group commutator isHere and big O notation refers to any fixed matrix norm. Active rotation matrices act on column vectors from right to left: first rotate about by , then about by , then about by , and finally about by . Swapping and replaces the product by its inverse and changes the leading Lie bracket to .
Writing and , direct matrix multiplication givesFor radians, , whereas . Thus the residual is a positive rotation about to leading order; at finite it also has smaller components. The Lie bracket captures the leading effect of the four moves rather than their entire finite motion. See Etingof, Section 7.2, Proposition 7.11.
The left panel projects the actual four arcs onto the -plane for ; the start and endpoint are close but distinct. The right panel divides the residual coordinates by , making the limit visible. This is a numerical picture of in the SO(3) Lie algebra.
Stable digit sort 2026-10-18
A stable digit sort applies stable sorting to complete sorting records using one specified numerical digit of their encoded sort keys. Counting sort can supply it; stable record placement in counting sort moves records without discarding their associated data. Stability concerns equal current numerical digits, even when the full sort keys differ. This is the information-preservation step in least-significant-digit radix sort.[ref]
11.2 Counting Sort and Radix Sort. Open Data Structures: An Introduction by Pat Morin; §11.2.1, associated-data and stability paragraphs; §11.2.2, least-significant passes and correctness paragraph after Figure 11.8
An original two-record counterexample exposes the need for the tie rule. Start with at width three. A units pass gives . Both tens numerical digits are four. A deliberately unstable tens pass may reverse their tie and return : it is sorted by the tens numerical digit, yet its two-position suffixes are decreasing. The final stable hundreds pass sees two zeros and keeps this wrong order. Sorting permutation preservation still holds, but sortedness fails. Stability in the tens pass would have retained the units information.
With counting sort as the stable digit sort, each pass initializes counters, counts current numerical digits, constructs their prefix sums, and places complete sorting records. Thus each pass costs , and passes cost in Big O notation. Reuse the counters and two arrays of record references between passes: auxiliary space complexity is , rather than . Assume constant-cost numerical digit extraction, counter arithmetic, array indexing and record-reference movement; the bound counts these operations, not the cost of copying arbitrarily large payloads or performing arbitrary-precision integer arithmetic.
The benefit is avoiding a counter for every possible whole sort key: width has possibilities, while each pass needs only counters. A larger radix reduces the number of passes for a fixed key range but enlarges the counter array. The comparison sorting lower bound does not apply to these direct array indices; they reveal more than a two-way comparison. Nor does fixing make disappear: growing key width still increases work.
Stable record placement in counting sort 2026-10-18
Initialize a mutable endpoint array to the inclusive cumulative counts . Scan input array indices . For each complete sorting record with sort key , first decrement , then write at output array index . This is equivalent to writing at the old endpoint minus one and then decrementing the endpoint.[ref]
11.2 Counting Sort and Radix Sort. Open Data Structures: An Introduction by Pat Morin; §11.2.1 Counting Sort, final scan paragraph and stability paragraph after Theorem 11.7
In the worked example, the first input sorting record read is at array index six. Its endpoint changes from to , so the complete sorting record goes to . Next, changes from seven to six and goes to . Writing before the decrement at the old endpoint would put outside its reserved integer interval and try to put beyond the output array. All seven writes are:Each endpoint changes only when a sorting record with its own sort key is placed. Consequently, stays at five through the placements of before dropping to four for .
Here is a complete implementation under the stated input contract; each sorting record is a pair whose first component is its integer sort key:
def counting_sort_records(records, K):
end = [0] * K
output = [None] * len(records)
for key, data in records:
end[key] += 1
for key in range(1, K):
end[key] += end[key - 1]
for i in range(len(records) - 1, -1, -1):
key = records[i][0]
end[key] -= 1
output[end[key]] = records[i]
return outputThe same array
end successively stores frequencies, inclusive cumulative counts, and moving exclusive endpoints; the formulas describe their mathematical values without requiring three stored arrays. The write copies the complete sorting record, including its associated data. In particular, two occurrences of an identical pair still cause two separate writes.For a short algorithm correctness proof, let be the number of already placed sorting records with sort key . A loop invariant isAll integer intervals here mean output array indices. Initially , so each whole reserved integer interval is unfilled. When the next sorting record has sort key , fewer than such occurrences have been placed, giving . Decrementing selects a fresh slot inside its reserved integer interval. At termination, and : every output slot has been filled exactly once with a complete input occurrence. The disjoint, increasing sort key ranges establish sortedness, while the fresh-slot property establishes sorting permutation preservation.
For stable sorting, take two input occurrences with equal sort keys at array indices . The backward scan visits first and puts it farther right in their shared integer interval. When is visited, the endpoint has decreased at least once, so its output position is smaller. Thus their original order survives. A forward scan with this same decrement rule reverses every equal-sort key group: it would put at positions five, four, three. A forward scan can instead be stable if it starts from and fills each integer interval left to right, incrementing after each write. The scan direction and endpoint convention must agree.
Count initialization takes writes, output initialization takes writes, counting takes updates, the prefix sum pass takes updates, and placement takes writes. Hence time complexity and auxiliary storage are in Big O notation, assuming constant-cost sort key access, integer arithmetic, array indexing and complete sorting record movement. Large records can instead be represented by constant-size references.[ref]
11.2 Counting Sort and Radix Sort. Open Data Structures: An Introduction by Pat Morin; §11.2.1 Counting Sort, auxiliary counter and output arrays, loop-cost discussion and Theorem 11.7
For example, three sorting records with sort keys zero, and zero require one million counters if ; unused sort keys still incur initialization and prefix sum work. The bound is therefore linear in the input length only when . Empty input skips the record scans, but this implementation still initializes counters and accumulates their prefix sums. For nonempty input, counting all records and filling their ranges gives a concrete termination argument.
Direct sort key indexing is a different operation from a binary comparison. The comparison sorting lower bound concerns comparison sorts; counting sort uses its restricted integer sort keys as array indices. Its small- speed is particularly useful for categorical numerical labels and for each stable sorting pass of radix sort: the reserved integer intervals turn group sizes into exact destinations.
Tangent plane Created 2026-10-14 Updated 2026-10-31
For an embedded regular surface and , the affine tangent plane isIt is a set of points through , whereas the tangent space is a vector space of displacements with zero vector. Some texts also call that two-dimensional tangent space the tangent plane; here the translation by distinguishes the two uses. For a regular parametric surface with ,where the partial derivatives are evaluated at . Regularity means that these tangent vectors are linearly independent, equivalently . Their cross product is a normal vector. [ref]
https://www.math.uga.edu/sites/default/files/inline-files/ShifrinDiffGeo.pdf. Theodore Shifrin, Differential Geometry: A First Course in Curves and Surfaces, Chapter 2, §1: regular parametrizations and tangent planes
For an implicit regular surface , require a smooth function with . ThenDifferentiating proves that every surface velocity lies in the displayed kernel of a linear map; the implicit function theorem gives a local graph with two independent coordinates, proving equality. The gradient is the normal vector with respect to the ambient dot product. [ref]
For the sphere , , and the affine tangent plane has equation . At the unit sphere's north pole , compare the plane with the displacement tangent space . The regular parametric surface , for , gives and , confirming the same answer.
This analytical illustration shows a unit sphere and a vertical cross-section. The smooth curves and give the two displayed tangent vectors at . The straight point for any nonzero is outside the sphere when , since . A tangent plane therefore captures first-order motion, not an exact patch of the regular surface.
The same idea makes local approximation useful. For the graph of a function with a smooth function, the affine tangent plane is . [ref]
https://openstax.org/books/calculus-volume-3/pages/4-4-tangent-planes-and-linear-approximations. Gilbert Strang and Edwin Herman, OpenStax Calculus Volume 3, §4.4: tangent planes and linear approximations
With continuous second partial derivatives, the vertical error is in Big O notation. On the unit sphere near its north pole, : the tangent plane loses quadratic bending while preserving all first-order velocities.A failed regularity test needs interpretation. Replacing the sphere equation by makes everywhere on the same perfectly regular sphere, so this defining smooth function cannot recover its tangent plane. Similarly, the usual spherical parametric surface loses matrix rank at the poles, while the graph parametrization above remains regular there. A vanishing gradient or degenerate parametrization invalidates that calculation; it alone does not prove that the underlying geometry has no tangent plane.
Tidal force Created 2026-10-06 Updated 2026-10-14
Differential gravitational acceleration, multiplied by the affected mass, across a spatially extended body. In the linear approximation the acceleration difference is , where is the tidal tensor. Stellar tidal forces limit the domain of a circumplanetary orbit; sufficiently strong ones can produce tidal disruption.
To isolate a tidal force, subtract the gravitational acceleration shared by the body as a whole. Let a reference test particle follow , and let another have position . In a translating accelerated reference frame with parallel, nonrotating axes,for a source point mass, when . Here is its standard gravitational parameter and the last term uses Big O notation. This Taylor expansion removes the common inward pull and keeps the spatial variation. The near side has an excess pull toward the source; the far side has a deficit of inward pull, which is an outward relative acceleration. Both sides can stretch apart while both still accelerate toward the source.[ref]
Chapter 7: The Theory of Gravitation. §7–4, discussion of stronger near-side and weaker far-side attraction
Subtracting common gravitational acceleration reveals radial stretching and transverse squeezing
. Blue arrows show the exact Newtonian gravitational field at several positions near a reference point. Orange arrows show its linear residual, calculated with the tidal tensor. Arrow lengths use separate scales in the two panels; this is a comparison of direction and spatial variation, not two equal-magnitude fields. Details.For a material element of mass , the residual external force in this translating accelerated reference frame is . Actual deformation also depends on elasticity, pressure and self-gravity. A rigid body approximation suppresses deformation but can still have internal forces that resist the tidal force. The point-mass tidal stretching and squeezing pattern describes freely falling, noninteracting test particles, rather than promising the same shape for every solid, star or ocean. A rotating reference frame introduces centrifugal acceleration and Coriolis acceleration, which must be included separately.
Time complexity Created 2026-09-24 Updated 2026-10-14
The time complexity of an algorithm bounds its number of computation steps as a function of its input length.
Specify both the size parameter and the operations counted. For binary search, take to be the number of array entries and count a three-way comparison of the target with one entry as one operation. Code using separate equality and less-than tests may perform two primitive tests per inspected entry; this changes a constant factor. The input size in bits is a different parameter when entries have variable-length representations.
Worst-case time complexity maximizes the count over permitted inputs of size . An absent target forces comparisons in linear search. For binary search, the worst-case count is for , attained by a target larger than every entry. Both have a best case of one comparison when the first inspected entry matches.
In Big O notation, the worst-case bounds are and respectively. Here means that fixed constants exist with for all . It is an upper bound, so a logarithmic count is also ; the sharper bound communicates its scaling better. The base of the logarithm changes only a constant factor.
For , linear search can require comparisons, whereas binary search needs at most , since . Doubling adds just one to the worst-case binary search count. These are operation counts, not measured machine runtimes. Translating them into logarithmic execution time assumes constant-cost indexed access, comparison and index arithmetic. Long entries can make comparison expensive; a linked list does not provide constant-cost access to an arbitrary middle index. Building or sorting the array is additional work: merge sort can provide order for many subsequent searches, but its cost belongs in the analysis too.
Trace and local volume change Created 2026-10-14 Updated 2026-10-20
For any fixed real -by- matrix , as the real parameter tends to zero,The big O notation means the remainder is bounded in absolute value by for some constant near zero. The determinant stays positive for sufficiently small , so its oriented volume factor is also its ordinary volume factor. Thus the trace is the first-order fractional volume change, measured from the identity matrix.
A proof follows directly from multilinearity of the determinant. Each column of is . The constant term chooses all the standard basis columns and equals one. A term containing exactly one factor replaces just column by ; expanding that vector in the standard basis leaves only its component, , because every other choice repeats a column. Summing gives . Terms with two or more replacements have order or higher. This also proves the derivative of the determinant at the identity matrix.
This coefficient has a direct visual meaning. At the point , the linear map has component along direction . In a small step , these are the first-order changes of the square's or box's sides in their own directions. Summing them gives its fractional area or volume change; the off-diagonal entries initially change shape rather than contributing to this first-order total.
For the constant-coefficient linear system of differential equations , the exact linear flow is , using the matrix exponential. Its volume factor isThis is the matrix exponential determinant identity. One can derive it without diagonalization: set , whose matrix inverse is , and use in Jacobi's formula to get . The scalar differential equation with initial value one gives the displayed exponential function. Thus positive trace causes volume expansion, negative trace causes contraction, and zero trace preserves volume, even when individual directions behave differently.
In two dimensions, compare four generators at :The first and third diagonal matrices expand or contract both directions. The second expands horizontally and contracts vertically by reciprocal factors, making a rectangle of unchanged area. For the last nilpotent matrix, , so its matrix exponential is exactly ; it sends to , a shear mapping. Zero trace allows substantial shape change and does not mean the flow map is stationary.
Four examples of a linear system of differential equations: trace controls area change while a saddle and a shear mapping preserve area
. Details. The first-order statement and the exact flow map should be distinguished. For , , whereas for all real . Zero trace eliminates the first-order change in the approximate step, and it preserves volume exactly in the matrix exponential flow. The examples use an abstract parameter and coordinate vectors; no physical units are assumed.
Tree-structured constraint satisfaction 2026-10-25
A binary constraint satisfaction problem whose constraint graph is a tree. Restrict constraint domains by unary conditions first. Choose a rooted tree orientation and process children before parents. For each child of , retain in only values having a compatible value in the already reduced . Thus every retained parent value has support in each child subtree.
If the root constraint domain becomes empty, no solution exists. Otherwise choose any root value, then recursively choose supported child values. A child-to-parent arc consistency pass need not enforce full arc consistency in both directions; the chosen directional condition suffices for this construction.
The proof uses mathematical induction on subtrees. A retained leaf value satisfies its unary requirements. Each retained parent value has a supported value in every child subtree. Those subtree solutions can be combined because the tree has no edges between different child subtrees. Therefore every retained parent value extends to a whole subtree solution, and every retained root value extends to a complete solution without backtracking.
For variables and maximum constraint domain size , scanning all value pairs once per edge costs time complexity in Big O notation, when one binary constraint evaluation has unit cost. If a constraint requires summing bundle values, its evaluation cost must also be counted.
Tridiagonal matrix algorithm 2026-10-14
The Thomas algorithm is Gaussian elimination specialized to a tridiagonal matrix: each row couples an unknown only to its immediate neighbours. Eliminating one unknown changes just the next diagonal entry and right-hand side. Provided all pivot elements are nonzero, a forward sweep followed by back substitution solves an -unknown linear system in time and storage, using Big O notation. Treating the same entries as a full matrix wastes the zeros: classical dense LU factorization and solve costs grow cubically for factorization and quadratically for storage.
A concrete reason to care is the Dirichlet Poisson solution by tridiagonal elimination: a three-point finite difference for the second derivative couples adjacent points on a mesh, producing precisely this tridiagonal matrix. The same structure occurs in one-dimensional Crank-Nicolson method calculations. NPTEL's archived Lecture 13 introduces the Thomas algorithm from that time-stepping problem and labels the procedure “Thomas algorithm (1949)”.[ref]
Thomas algorithm (NPTEL Lecture 13). Lecture 13, Thomas algorithm: opening historical attribution and Crank Nicolson motivation
Union-find operation bounds 2026-10-18
Assume identifiers, pointers and ranks fit into constant-size machine words, with constant-cost indexed array access, comparison and assignment. Initializing members costs time and machine words. With union by rank and full path compression, any sequence of disjoint-set find operations and disjoint-set union operations then has total time complexity, in Big O notation,Here is a conventionally normalized inverse Ackermann function. The documented path-halving implementation has the same amortized analysis bound.[ref]
UF.java. Algorithms, 4th edition by Robert Sedgewick and others; UF.java, class documentation: linear constructor, logarithmic worst-case operations and inverse-Ackermann sequence bound for rank with halving
A disjoint-set union operation includes two disjoint-set find operations and a constant amount of additional linking work. Including in the bound accounts for initialization even if no queries follow.
This is a bound on an entire sequence. It is not a worst-case constant bound for each call: a disjoint-set find operation can still encounter a previously uncompressed route of logarithmic length. The inverse Ackermann function grows extraordinarily slowly but is unbounded, so it cannot be replaced by a mathematical constant in this asymptotic statement. The general distinction between a per-operation maximum and a whole-sequence guarantee is explained in amortized analysis.
Robert Tarjan's 1975 paper established an inverse Ackermann function bound for set union. Its abstract states the sharper two-parameter form for representative queries and intermixed merges: the worst total cost is bounded above and below by positive constant multiples of . Those variables count different operations from the convenient formulation above; the abstract's matching lower bound is about its analyzed method, rather than a claim that every conceivable representation obeys the same lower bound.[ref]
The intuition is that union by rank limits how parent routes grow, while path compression makes an expensive traversal improve later queries. The rank argument alone proves only a logarithmic bound. Establishing the inverse Ackermann function bound requires a finer accounting of traversals across rank ranges; it does not follow merely from drawing a flatter rooted tree. The useful result is that repeated merging and membership queries can share the work of maintaining a changing partition of a set.






