This paper presents MPIAB, an agent based architecture for parallel processing. The architecture is developed to model the functions of standard MPI using Java agents. It remedies the deficiencies that exist in MPI, including the incapability to operate in heterogeneous environments. The architecture also addresses other issues such as effective utilization of system resources by dynamically selecting the least busy computing nodes through the computation of a threshold barrier value. Our proposed agent architecture would integrate the power of Java technology and agents to support efficient communication and synchronization of the nodes over the network for parallel processing.