Man ! Your story is almost the same as mine. I left office 10 years ago maybe not for AJ because I didn’t know that this site exist, but for music and playing guitar. It was hard time for me. After year of stagnation, complaints from my family and all bullshit which musicians must listen from “wiser” people I got a job in a music school. Moreover month later I joined cover band , you know - weddings, events etc. When my family,friends, and former office co workers realized that I earn more than them, they finally shut up 
2 years ago I discovered Audiojungle and stock music business and I fell in love with composing , mixing and overall production. I’m still learning but I believe that I will be good at it. If you work hard there is no other way 
Today I know that it will be my main job. I don’t want to play weddings, events whole life and music which I don’t like. I prefer rock, metal and I love cinematic music. This site allows me to share my ideas and moreover to earn money.
So friend. Keep going. It’s good to know that I’m not alone with my problems and decisions
One important advice - don’t listen to other people especially who know nothing about music, stock music, production etc. They think that we make these “stupid” melodies for fun and working at home it’s not a real work Sounds familiar ? I believe that many of us heard that bullshit
Sorry for making an essay in your topic but you described so many things that also concern my person and I think many other authors here as well.
To sum up - I wish you all the best here and in whole music career