I'd disagree with the filler thing. Most of the book actually has an overarching plot. There are some instances which are incorporated as imagery meant to foreshadow future events. Also, the trial isn't the main focus of the book. It's certainly a huge part of it, but it's only a part of the books message on judgement. The old lady, Tom Robinson, Boo Radley, the farm boy they invited for dinner. The whole thing is about judgement.
Then again, it's actually one of my favorite books.