Based on the regular and symmetrical reinforced concrete typical spatial frame, 8 kinds of infill walls arrangements were selected out for comparison including plane and/or vertical irregular configuration. Based on the OpenSees framework, flexibility method based fiber model was adopted to simulate the nonlinear responses of columns under biaxial bending and varying axial forces, and equivalent model with diagonal compression strut was adopted to simulate the infill walls. Nonlinear responses analyses of the each spatial frame under rare earthquake excitations were carried out. The results showed that reasonable arrangements of infill walls could effectively decrease seismic responses of spatial frames. Vertical irregular configuration of infill walls will significantly increase the seismic response of frames, and story side sway mechanism always develops at the floor that infill walls were lessened or abolished. Plane irregular configuration of infill walls occurring similarly at every floor could decreases the seismic responses of frame to a certain extent, but plastic deformations will concentrate on some local plane locations where the infill walls were abolished. Plane and vertical irregular configuration of infill walls will also result in the plastic deformations concentrating on specific floor, the effects of concentration of plastic deformations caused by vertical irregular configuration will remarkably surpass the adverse effect of nonlinear seismic responses due to plane irregular configuration of infill walls.