I had a strange issue when deleting a Drupal installation. The sites/default/ directory has funky permissions belonging to the Apache user. The result is I cannot delete the directory and files within it. The solution was to SSH into my web host (on a shared account) and execute the following commands.

chmod -R 0777 sites/

rm -rf sites/