AstroPhot quickly extracts detailed information from complex astronomical data for individual images or large survey programs. It fits models for sky, stars, galaxies, PSFs, and more in a principled chi^2 forward optimization, recovering Bayesian posterior information and covariance of all parameters. The code optimizes forward models on CPU or GPU, across images that are large, multi-band, multi-epoch, rotated, dithered, and more. Models are optimized together, thus handling overlapping objects and including the covariance between parameters (including PSF and galaxy parameters). AstroPhot includes several optimization algorithms, including Levenberg-Marquardt, Gradient descent, and No-U-Turn MCMC sampling.