Before you get too overwhelmed, let's start with the basic definition of onomatopoeia. According to the CliffsNotes Literary Terms & Poetry Glossary, onomatopoeia is:
The use of words whose sound suggests their meaning. Examples are "buzz," "hiss," or "honk."
In other words: written sound effects. I'm not sure what "onomatopoeia" is supposed to sound like, but it does sound odd — so maybe that can help you remember what the term means.
It's true that some instances of onomatopoeia are easier to identify than others. The most obvious examples stand out like a fight scene from an old TV episode of Batman: Ka-pow! Bonk! Zing! However, chances are that the poems you study in your literature class employ a subtler type of this device.
Poets usually intend for their work to be read out loud, so they choose their words based on the way they sound. Sometimes that means rhyming words, or alliteration (repetition of a particular consonant), or assonance (repetition of a particular vowel sound) — or onomatopoeia. When a poet uses onomatopoeia, the sound effects you hear will usually reinforce the overall theme of the poem. Often, this device shows up not as a single word, but as a cumulative effect. For example, instead of using the obvious sound-effect word buzz to represent the sound of bees, Tennyson uses several words with a repeating mmm sound to create a bee-like effect in Come Down O Maid:
The moan of doves in immemorial elms,
And murmuring of innumerable bees
You wouldn't ordinarily think of immemorial is an example of onomatopoeia, but it becomes one when used in this clever way. So, rather than trying to memorize a list of all the possible onomatopoeia words, instead practice listening for the sound effects as you read a poem (especially if that poem is about something that makes noise!).
Here's one more famous example of onomatopoeia. Read this excerpt from The Bells, by Edgar Allan Poe, out loud and listen for how the words create all sorts of bell-like sounds:
Hear the sledges with the bells —
Silver bells!
What a world of merriment their melody foretells!
How they tinkle, tinkle, tinkle,
In the icy air of night!
While the stars that over sprinkle
All the heavens, seem to twinkle
With a crystalline delight;
Keeping time, time, time,
In a sort of Runic rhyme,
To the tintinnabulation that so musically wells
From the bells, bells, bells, bells,
Bells, bells, bells —
From the jingling and the tinkling of the bells.
Cliffs Notes Online