Could it be?

How much the genetic test costs depends on whether you have insurance that will pay. My understanding is that without insurance, in the U.S. the test costs around $3000. 

Normally it's preferable that someone in the family who has had cancer gets tested first.  Have any of your relatives been tested?  If any of them are BRCA positive, that provides a rationale for you to be tested (making the insurance approval easier to get, if you have insurance) and it makes the testing less expensive because they know exactly which BRCA variant they are looking for when they test you.

What about your mother?  The fact that you have a number of 2nd and 3rd degree relatives possibily suggests that you have an increased risk, but it's really first degree relatives that are most important.  Here's why:  From the sound of it, it might be that the BRCA genetic mutation is in your family.  Perhaps one of your maternal great grandparents had the mutation, and passed it along to your grandmother and great aunt.  Your grandmother in turn passed it along to your two aunts.  But maybe your mother didn't inherit it.  This particular gene is inherited either from one's mother or one's father.  So even if one of our parents has the BRCA mutation, there is only a 50% chance that we will inherit it. Your mother might have inherited this gene from her father rather than from her mother.  This would mean that although your grandmother and aunts are all affected and are very high risk, your mother would not have any increase in risk vs. the average woman.  And if your mother doesn't have the BRCA mutation, then you can't have it either. It doesn't skip a generation - there needs to be a straight line of inheritance.

So from your family history, if your mother has not had either breast or ovarian cancer or any cancer related to the BRCA mutation, then it sounds as though you have about a 25% chance of being high risk - assuming that the mutation is in fact in your family.  If your grandmother has the mutation, your mother had a 50% chance that she inherited it, and if she did, you had a 50% chance of inheriting it from your mother.  50% of 50% = 25%.

All that to say that despite your strong family history, your risk level could be completely normal and not increased at all.  That's not to say that you shouldn't pursue testing, but that might help explain why it makes more sense for your aunt or grandmother to be tested first.  Find out if the BRCA mutation is in the family, and then go from their with additional testing, if necessary.

As for what's involved, I had just a simple blood test.  I believe these days they also do a cheek swab or saliva or something like that.

What will it tell you?  If you are positive, you will have an increased risk to develop breast cancer and ovarian cancer and possibly other cancers.  The actual risk level depends on the specific BRCA mutation that you have (there are hundreds of different ones).  The test will not tell you if you will develop these cancers, just that you are higher risk.  Some women with a BRCA mutation don't develop cancer.

If you test negative, given your family history, it won't tell you much unless you know that the BRCA mutation is in your family.  If your grandmother is tested and found to be BRCA positive and then your mother or you are tested and found to be negative, that will provide a clear answer.  You will know what is causing all the breast cancer in your family, and you will know that you do not carry that same genetic mutation. On the other hand, if your grandmother is tested and found to be negative, then there is no point in you being tested, because there is nothing to find.  What that would mean is that there could be another as yet undiscovered gene that is causing all the cancer in your family.  So this too is another reason why someone who has had breast cancer in your family should be tested first.  If you are tested without any other information about your family, a negative test result won't be very helpful.

Have you talked to a genetic counsellor?  Usually that's the first step.  They would go over your family health history and recommend if you - or someone else in your family - should be tested.  There are other less common genetic mutations that can also increase breast cancer risk and by going through your family history and looking at all the different types of cancer that both males and females have had, the genetic counsellor would determine if the testing should be done just for the BRCA mutation or if the testing should include other genes too. 

Talking to a counsellor would be my recommendation as a first step.