Polyethylene pipe has been used widely in gas transportation and nuclear safety-related cooling water applications due to its exceptional resistance to corrosion and erosion. Butt fusion joint is one of the main welding forms for polyethylene pipes. Ultrasonic technique is a typical nondestructive examination technique. To overcome the coupling problems when inspecting butt fusion structures, an inspection technique of ultrasonic phased array using water wedge is proposed to solve the coupling matching and to increase the ultrasound amplitudes. The influences on the imaging of the parameters (such as the angle and height of the water wedge), the array element specifications, and the aperture, were investigated via simulations. The parameter optimization was conducted to establish a suitable detection process. After that, the related probe and a simulated water wedge were designed based on the results and manufactured, in which the simulated wedge as a specific probe holder could adjust its own incident angle and array height. Meanwhile, a typical DN315 pipe of PE 100 was made with some typical artificial defects in it. Experiments were conducted, and the results showed that the proposed water-wedged ultrasonic phased array technique is suitable for butt fusion joint inspection.