Yes, you may delete unwanted cards by typing command-space (to summon the menubar) and then choosing "Delete Card" from the edit menu. Some cards will react in special ways. Island Quiz cards want to stay in groups of six and will give you a choice of deleting all or none (choose none!)

Generally speaking, it's better to refrain from deleting cards. I will cheerfully clear away any unwanted cards at Harvest time. Deleting cards can sometimes be dangerous whereas it's almost never dangerous NOT to delete a card.