calculate a third column based on the dates in A. you can use a formula like =MONTH(A1) + "/" + YEAR(A1) to extrace just the month and year. then create a pivot table (it's under the data menu) where you choose this month column and column B. you can put the month column on either the X or Y axis (depending on if you want your averages to go across or down the page), and put column B should go into the center of the table. by default, the pivot table will try to just do a SUM for each month, but you can select the field properties on the pivot table toolbar and ask it to display averages instead.
edit: instead of the formula i posted to extract the date, you may be better of creating a new date based on only the month and year and something like day 1 (like this: = DATE(YEAR(A1), MONTH(A1), 1) ). that way you have a date rather than a string. you can always format the date to show only the month and year at that point.