This paper presents a novel smoothing gradient damage model, which goes beyond certain limitations of conventional methods, for accurate prediction of localized failure in quasi-brittle materials. The proposed method is particularly tailored to low-order finite elements such as 4-node quadrilateral or 3-node triangular elements. The low-order elements are preferable in practice as they can automatically be generated for problems even with complex geometries at low computational cost. In order to eliminate spurious damage growth and correct wrong prediction of shear band induced by using the constant gradient parameter in terms of conventional models, we thus introduce a novel modified evolving anisotropic nonlocal gradient parameter, which aims to control the behavior of nonlocal interactions of damage microprocess during the entire loading history in a more appropriate manner. Unlike conventional approaches, the novel modified evolving gradient parameter heavily depends on the principal stress and equivalent strain states, which serves to reduce the impact of localized deformation. The stress fields thus play a crucial role in the present formulation as they greatly affect the orientation and intensity of nonlocal interactions. The quality of raw stresses becomes critical (e.g., stress oscillation) once two unknowns (i.e., displacements and nonlocal equivalent strain) are approximated simultaneously using the same orders of interpolation functions (e.g., linear–linear). A smoothing technique is thus adopted to smooth out the raw stresses. The stresses after smoothing are shown adequately in the estimation of the new gradient parameter, providing much better solutions. In addition, to precisely capture the softening in quasi-brittle materials, the original energy norm is decomposed into tensile and compressive parts to form a new bi-energy norm. The scalar equivalent strain is thus estimated through this new bi-energy norm, which is obviously able to distinguish tensile and compressive conditions. To further enhance the capability of the present damage model, the material softening process is also determined through fracture energy in terms of fracture mechanics. Comparison of the present results with reference solutions derived from experimental data and other numerical methods for benchmark applications, regarding the nonlocal equivalent strain, damage profile, structural force–displacement curves, etc., confirms the accuracy and superior performance of the proposed approach for characterizing localized failure in quasi-brittle materials.