An official told Reuters the militants had occupied 75% of the Baiji refinery, 210km (130 miles) north of Baghdad.
The army said 40 attackers had been killed, a claim which could not be verified independently.
Prime Minister Nouri al-Maliki has gone on television to urge Iraqis to unite against the militants.
Government forces are battling to push back ISIS (Islamic State in Iraq and the Levant) and its Sunni Muslim allies in Diyala and Salahuddin provinces, after the militants overran the second city, Mosul, last week.
In other developments:
Militants 'in control'
The attack on the refinery started at 04:00 (01:00 GMT) from outside two of the three main entrances to the refinery, according to Reuters.
"The militants have managed to break into the refinery," the unnamed official told Reuters from inside the refinery. "Now they are in control of the production units, administration building and four watchtowers. This is 75% of the refinery."
Army spokesman Qasim Ata said in news conference broadcast live on TV: "The security forces thwarted an attempt by ISIS to attack the Baiji refinery and 40 terrorists were killed."
The nearby town of Baiji was overrun by ISIS-led militants last week. Foreign personnel, including a small number of British nationals, were evacuated from the refinery earlier but local staff reportedly remained in place.
Baiji accounts for a little more than a quarter of the country's entire refining capacity, all of which goes toward domestic consumption for things like petrol, cooking oil and fuel for power stations, an official told AP news agency.
Hundreds of people have been killed since the start of the militant offensive last week, many of them believed to be captured soldiers publicly shot by ISIS-led firing squads.
During fighting in the city of Baquba this week, 44 prisoners were killed inside a police station in unclear circumstances.
'A setback'
Government forces have renewed air strikes on militants while militants in the western province of Anbar say they have made advances, with a number of police stations near the town of Hit going over to dissident tribes.
Further north, the Iraqi government said it had recaptured the citadel in the strategic town of Tal Afar, where militants were said to have taken control on Monday.
An army spokesman told the AFP news agency troops were planning to press on to militant-held areas in Mosul on Thursday.
"I would like to say once again that what has happened in Iraq is a setback but not every setback is a defeat," Mr Maliki said on Wednesday.
"This setback has allowed Iraq to recover its national unity and Iraqis have managed to recover their feeling that they are in danger and that not a single Iraqi will benefit from this crisis."
Mr Maliki has long been accused of favouring the country's Shia Muslim majority and fomenting unrest among the Sunni minority.
Iranian President Hassan Rouhani has said Tehran will not "spare any effort" to defend Shia holy shrines in Iraq against "mercenaries, murderers and terrorists".
He was speaking amid reports that the head of the elite Quds Force of the Iranian Revolutionary Guards, Qasem Soleimani, was in Baghdad to help co-ordinate the fight against the militants.
ISIS in Iraq
ISIS grew out of an al-Qaeda-linked organisation in Iraq