Is Highs and Lows about Kanye's fans?

(stupid question but i needed to know)

Highs and lows:- is what he went through high being his start and lows being current.

I put you through a lot I know:- meaning he put us through a lot by the statements he has said and all

But still you didn't let me go :- meaning that we didnt leave him and hoped he would come back to his original self?