Duct Removal Surgery or Wait?

Since I had kids I have had a discharge from both breasts and multiple ducts on each when expressed (doesn't leak spontaneously). It was normally a yellowish milky color and slightly sticky. I check the discharge from time to time to make sure it is still there and see if there are any changes. I had a routine Mammogram in Aug of last year which was normal. Around October when checking the discharge I noticed that for one duct on my right breast the discharge had changed in color. It was now a clear black color when expressed, but when put on a tissue looks more blue. All the other ducts were pretty normal color but does have a slightly darker tinge and is a bit more sticky than normal. This has continued for several months so at my regular checkup I mentioned it to my primary care physician. I showed him the discharge and he took a sample of it using a syringe and sent it to the lab. The following were the results:

FNA RIGHT BREAST
SCANT DUCTAL CELLS ARE PRESENT.
SCANT FOAM CELLS ARE PRESENT.

No white blood cells seen.

One cluster of duct epithelium is seen, a few foamy cells, and amorphous
debris.

The specimen is scant, but the presence of duct epithelium in nipple
discharge suggests a ductal lesion, most often benign.

When he called me with the results he just said it is benign but there were some abnormal cells and he was referring me to the surgeon (I later looked at the results in my record and captured the above).

I met with the surgeon and showed him the issue. He tested the discharge with some strips (I think looking for blood) and said it was fine and then referred me to get a ductogram. Since I am hypothyroid as well and my thyroid numbers have been way off lately and I mentioned my concern to him, he also ordered a blood test to test my prolactin levels.

When I went to the center for the ductogram they did a new mammogram and then did an ultrasound of the right breast. They only found one patch of dense tissue that the Dr. here said had no definition and was most likely connective tissue and not of concern (it was also on the opposite side of the breast from the duct having issues). Then they did the ductogram and after looking at the image said he saw nothing of concern (and even kept commenting on what a great image they got).

I followed up with the surgeon today who said something is causing the discharge (my PCP had mentioned that possibly the changes are from hormones due to starting menopause) and I have two options. When I asked if hormones would cause it he said no as that wouldn't affect just one duct. He said prolactin levels were normal. He said we can either wait and monitor it and see if there are any further changes and get a followup mammogram/ultrasound in a year. Alternately, I could have the duct surgically removed and they would biopsy it. He mentioned that many people get it removed so they know for sure but he did say it was most likely nothing to worry about and benign.

I'm just looking for advice. Has anyone had the duct removal and how invasive is it and and how is recovery? Dr. had said if I had it on Wed I would be back at work on Monday. I hate to elect a surgery and that risk (I have a lot of risk factors such as diabetes, sleep apnea, etc) when it is likely nothing to worry about. Why would they want to remove it if the cells were benign? If I wait, what are the risks and concerns and what should I be looking out for? Has anyone gone through something similar and what was your outcome?