The Glovers who took the lead in the 13th minute when Cousins stabbed home from close range after the Millwall defence failed to clear their lines following a Helen Bleazard free-kick.
On 21 minutes, Ellie Curson had an angled shot turned behind for a corner by Lionesses’ keeper Sarah Quantrill.
Lily Agg attempted a long range effort that was saved by Howard on 34 minutes as the Lionesses looked to find an equaliser.
In the second half, Yeovil made a bright start as debutant Sarah Wiltshire was denied by a one-handed save by Quantrall in the 52nd minute.
Wiltshire almost opened her account for the Glovers seven minutes later after she jinked through the Millwall defence, but was unable to find the target.
On 69 minutes, Yeovil keeper Howard produced a stunning one-handed save to tip a shot from Agg over the bar.
The hosts doubled their lead in the 73rd minute through a sweetly struck free-kick from Bleazard from the edge of the penalty area after Wiltshire had been fouled as they marked their return to WSL action with a victory.
Yeovil: (4-2-3-1) Howard, Cousins, Edwards, Haigh, Kirby, Radburn-Lorton, Pearson (Sherwood 83), Curson (Stewart 85), Wiltshire, Cook (Bray 89)
Subs not used: Whitton, Tewkesbury, Cossens, Brazell,
Millwall: (4-3-3) Quantrill, Cole, Dempster (Mackie 53), Davies, Strugnell, Rutherford, Maresevic (Lamb 64), Agg, Billson, Stenning (Albert 73), Perry
Subs not used: Albert, Maple, Tuison
Bookings: Billson (87)
Ref: S. Ogles
Yeovil face Watford on Saturday 2nd August at the Jones Stadium in a 6.00pm KO.
You can book tickets by visiting www.yeoviltownladies.com/tickets or by paying cash on the day costing £4.00 adults / £2.50 concessions.
Season tickets for all remaining games are also available, with adults costing just £12.50 and £5.00 concessions, which will allow entry to all 5 remaining home games this season. Visit www.yeoviltownladies.com for more info.