For a given stochastic process X, its segment Xt at time t represents the "slice" of each path of X over a fixed time-interval [t-r, t], where r is the length of the "memory" of the process. Segment processes are important in the study of stochastic systems with memory (stochastic functional differential equations, SFDEs). The main objective of this paper is to study non-linear transforms of segment processes. Towards this end, we construct a stochastic integral with respect to the Brownian segment process. The difficulty in this construction is the fact that the stochastic integrator is infinite dimensional and is not a (semi)martingale. We overcome this difficulty by employing Malliavin (anticipating) calculus techniques. The segment integral is interpreted as a Skorohod integral via a stochastic Fubini theorem. We then prove Itô's formula for the segment of a continuous Skorohod-type process and embed the segment calculus in the theory of anticipating calculus. Applications of the Itô formula include the weak infinitesimal generator for the solution segment of a stochastic system with memory, the associated Feynman-Kac formula and the Black-Scholes PDE for stock dynamics with memory.