BorderManager, WebSense and Abends...Oh My!

Last week a BorderManager server abended, nothing new there, happens. When it came back up, checked that the internet was there, and case over right?

Wrong....

The Surfcontrol filtering wasn't working, free porn and facebook for everyone!

1. The ruleset was restored from backup, no change
2. Removed the surfcontrol db and redownloaded it, no change
3. Management panic because actually letting the employees make decisions on time management is bad.
4. Call to the vendor
5. 2nd Call to the vendor
6. Email from vendor
7. Troubleshooting ensues
8. change this file, reboot, no change
9. change this setting, reboot, no change
10. reinstall software, reboot, no change
11. downgrade Bordermanager to 3.9 SP1, reboot, no change
12. see a pattern here?
13. reupgrade to BM3.9 SP2, but keep the proxy.nlm from 3.9 SP1, reboot, no change
14. Apply the BM3.9 SP2 IR1 patch, reboot, well the rules load quicker, but otherwise no change
15. Escalated to 2nd level support, repeat most of above, no change
16. Boss emails sales rep, sales rep calls support, escalated to the real geeks
17. 10 minutes of technical jiggery pokery, corrupted filtprod.dat found in sys\etc\border\english, replaced corrupted file, no reboot just issue a "stopbrd and a startbrd" and everything works!

I hate this sort of thing, if you work frontline support and can't fix it in ten minutes send it the next level please. Holding onto it may make you look like a hero, but you aren't doing much to inspire confidence in the users.