The buffer should only load new lines when scrolling up
Try moving that scrollbutton in the scrollbar to the top. Quassel will load new lines in the buffer. Then scroll down again and quassel will load some more lines without purpose. Perhaps it's better if it only loads after scrolling up.