As long as we're throwing out everything that's not falsifiable ...

One of the major scientific reasons for believing in the nonexistence of God is that he isn't falsifiable. That is, there's no experimental way to disprove his existence. Since it can't be disproven, it can't be proven, either.

In a sense that's true. God can't be proven or disproven by the scientific method. He doesn't show up just because we do an experiment. The mistake lies in taking that as evidence of his nonexistence.

There's plenty that can't be proven by the scientific method. "George Washington was the first president," can't be proven scientifically. You can't do an experiment to show that Santiago is the capital of Chile, that Oliver Cromwell was Lord Protector of England, or that anchovies taste bad. 

Or evolution, for that matter.

It's true; the quintessential scientific theory, and the basis for all modern scientific inquiry, is itself unscientific. After all, it's never been observed, either in the wild or in the laboratory. No experiment can confirm or deny it. In a word, it's unfalsifiable.

Some will say that we have a lot of evidence that evolution is true in the form of fossils. I would respectfully disagree. A fossil can only tell someone two things: that a thing used to be alive, and that that thing is now dead. Anything -- anything -- beyond that is speculation. It may be very well-reasoned speculation, but it's still speculation.

It's somehow scientific to believe one unprovable assertion and not another. I say take what science tells us, "Question Everything", and do it, even when science says not to. Be curious. One unprovable assertion will withstand it; the other won't.