The earliest documented mathematics comes from ancient Mesopotamia (Sumerians/Babylonians), roughly 3000 BCE, with cuneiform tablets showing base-60 arithmetic, multiplication tables, and basic algebra. Egypt follows closely behind, with the Rhind and Moscow papyri (c. 2000-1550 BCE) showing fractions, geometry, and arithmetic operations used for tax assessment, land surveying, and construction.
If you're drawing a distinction: counting and tallying predate both civilizations significantly. The Lebombo bone (Eswatini, ~43,000 years old) and Ishango bone (Congo, ~20,000 years old) are notched bones widely interpreted as early tally/counting devices, making them the oldest physical evidence of mathematical reasoning, though that's arithmetic in a rudimentary sense, not systematized number theory or written numeral systems.
"Modern arithmetic" as a formal system (base-10 positional notation, the zero placeholder, algorithmic methods) traces most directly to India (Hindu numeral system, codified by mathematicians like Brahmagupta in the 7th century CE) and was transmitted to the West via the Islamic Golden Age (Al-Khwarizmi, 9th century CE, whose name gives us "algorithm"). Europe adopted this as the "Hindu-Arabic numeral system," which is what's used globally today.
So: oldest evidence of mathematical practice — Africa (Lebombo/Ishango bones). Oldest documented systematic arithmetic — Sumer/Babylon. Foundation of the arithmetic system you're using right now — India, via Arab transmission.