The expression in the exercise is of the form (a−b)2, where a=y2 and b=3x2. With the help of the aquaring rule (a−b)2=a2−2ab+b2, we have
(y2−3x3)2=(y2)2−2y23x3+(3x3)2=y22−6x3y2+32x32=y4−6x3y2+9x6=9x6−6x3y2+y4