Prison workers in Victoria are at a "breaking point" due to toxic culture and poor pay and conditions, according to a public sector union report.