Annoying unintentional drag of the drawn box or line

[TishSerg]

When I have a diagonal line on my circuit it gets dragged whenever I click and drag in the rectangular zone around it (the rectangle whose diagonal is the drawn line). I expect to drag the line only when I click and drag the line itself (just like a wire). Not when I misclicked some component from any side of the line.
Same issue with the drawn box. I expect to drag it only when I click and drag its border, not when I misclick some component inside the box.