Telephone call that blew up in my face!

I am just wondering if anyone else has experienced this issue and if so, what did you do?

Chemo ended about one year ago and radiation about 10 months ago.  I’m on Herceptin and Tykerb.  About six weeks ago, I was run down and having difficulty dealing with work and life so I called the counseling and support "social work" line at the cancer clinic where I receive treatment.  After asking me a bunch of questions to "help figure out how to route my call," the social worker told me that they don't offer services to patients post treatment.  In other words, the call was a big waste of time.  She emailed me a list of referrals and I thought that was that.  In the meantime, I figured out how to sleep better, and generally started feeling better.

The next Herceptin treatment after I called the social worker line, my oncologist asked me about calling the line.  The social worker did not tell she was entering the call (and presumably the answers to her questions) into my file.  I told my doctor that I was run down but am sleeping better so I did not call any of the referrals.

Which brings me to today, the second Herceptin after I called the social worker.  My doctor asked me how I'm doing, I answered like I always do, that I'm tired/fatigued.  However, this time, she asked if I think it's from treatment or depression.  She also mentioned my call to the social worker line. 

So, now I feel that calling the social worker line was a big mistake and seems to be effecting how my doctor is hearing what I'm saying.  My doctor has no reason whatsoever to think I'm depressed other than the fact that I called this line and some note was entered into my file. 

I'm debating changing doctors.  Any advice?