How does a heart start beating?

A signal from the sino-atrial node. (More)

Do you mean as when the heart forms and begins beating for the first time ever, or do you mean if it has stopped?
If you mean the former, the heart begins beating on roughly day 22 of the child’s life, before the brain, liver, kidneys or nearly anything else has formed. Electrical signals are generated by the myocardium, basically the outer wall of the heart, and so the heart begins beating. more here

If you mean the latter, it can be stopped by interrupting the nervous impulses that travel to the heart or disturbing them, e.g. by severing the nerves or sending a strong enough current through the heart
(≥10 mA muscular contractions, victim cannot let go of wire
≈20 mA breathing becomes labored
by 75 mA, may cease completely
100–200 mA fatal, causes irregular twitching of heart walls (ventricular fibrillation)
≥200mA muscle contractions so strong that heart forcibly clamped by shock, doesn’t go into fibrillation
source)

Thanks @28lorelei. That’s what I was looking for. I found a similar answer on a forum after asking this.—I was looking for info on SIDS, with a connection to the heart.

@syz, your answer got me on the right path. Thank you.