Magnetoelectric properties are studied by a combined experimental and theoretical study of a quasi-two-dimensional material composed of square cupolas, Ba(TiO)Cu_{4}(PO_{4})_{4}. The magnetization is measured up to the field above the saturation, and several anomalies are observed depending on the field directions. We propose a S=1/2 spin model with Dzyaloshinskii-Moriya interactions, which reproduces the full magnetization curves well. Elaborating the phase diagram of the model, we show that the anomalies are explained by magnetoelectric phase transitions. Our theory also accounts for the scaling of the dielectric anomaly observed in the experiments. The results elucidate the crucial role of the in-plane component of Dzyaloshinskii-Moriya interactions, which is induced by the noncoplanar buckling of a square cupola. We also predict a "hidden" phase and another magnetoelectric response, both of which appear in a nonzero magnetic field.