It really depends on your prof. My Linear Algebra class was the hardest class I have ever taken. It wasn't the material that was hard, though, the prof that taught was, in fact, the devil, and made it hard on her students because she likes to torture people for all eternity, etc. Every assignment was the four to six hardest proofs in that section that she could find, her notes were a meandering mess of random thoughts, and the tests were over the application of Linear to real life. No one learned anything from her, you had to read the book and do other practice problems to understand the subject.
I later took a 3D computer graphics course, same material, and aced it. So it's all the prof, the teaching methodology that will determine it's difficulty. Good luck.