How to delete the settings record from the DB

[joveice]

How do you delete the settings record from the DB?
$user->settings()->clear(); leaves a empty array
$user->settings()->delete(); does nothing

When a user decides to delete his account I would like to remove the record from the DB.