I have that same dilemma, I love training and working out, but my goal is to be very slim. Overall I do feel like I'm going in the right direction for my goals. At the same time I do have moments when I feel like my arms look too muscular as opposed to lean or its hard to determine what's muscle and what's fat.
My aim is to be more lean than jacked. I'm fortunately a mesomorph so I can gain and lose fat/muscle whenever I please, but the fact that I vacillate so much probably has my body hating me. When at the gym I struggle over whether I should be lifting heavy to build muscle or do light weights with high reps to tone. I currently do a combination of both depending on my mood, I try to limit myself to one move with heavy weights per workout and the rest are medium to light or no resistance. And I'm finally overcoming my fear of cardio, which developed after I had a pretty intense over exercising point a few years back, which was a very rough time for me. I enjoy cardio a lot but its just reminding myself that I don't have to over do it.
There's a woman I follow on YouTube, her name is Stephanie Buttermore. She got extremely lean with a training routine, instead of skin and bone she was literally skin and muscle at her leanest. But she honestly kind of looked just as emaciated at as an Anorexic. I put a link to channel below. You can see from the preview images what I mean.
I started watching her because she's done these epic cheat day videos and it was very fascinating watching such a small woman put away that much food. But she has since "gone all in" to address her extreme hunger and she's gained a ton of weight on her journey, which is great for her because she has a different goal now. But a lot of her earlier videos are very inspiring to me.