The OP has already been negatively affected for a month and a half, he has a family history of needing ablation, and his primary already confirmed that he should see a cardiologist. I think it's safe to assume he's not panicked over a caffeine withdrawal induced day of palpitations.
Wait, you just said the referral by his primary is proof that he has something pathologic, yet previously said his primary is essentially worthless. His primary both knows it's serious and knows nothing? That doesn't seem to jive. Having a "family history of ablation" is quite vague and could imply any number of things, many of which may carry no significance to him as a patient whatsoever.
As for primary care... Even for a simple foot injury one gets forwarded over to an orthopedic. It's a wonder what primaries still do besides prescriptions and naming the specialist you actually need to go see. Medicine has grown so far beyond what one doctor can do.
You're speaking in some significant generalities. Unfortunately, there are a lot of bad primary care physicians out there who are simply referral mills, but identifying the profession by it's worst representatives is fairly disingenuous. I also, obviously, disagree with your last point. Certainly there is A LOT of medicine out there now, and it grows by the second, but it is hardly something that is marginalizing a good primary care doctor. The vast majority of it certainly can be managed by one well trained physician, especially if the environment were improved (e.g., administration forcing 10-20 minute visits per patient). This is not to downplay the need for specialists, there is a lot of value to good specialists when you get to the point that their services are required. However, there are just as many bad cardiologists, nephrologists, pulmonologists, etc. out there as primary care doctors.
What we need are empowered primary care physicians, rather than impaired ones.