How to Do Flexible Docking in AutoDock Vina (Flexible Side Chains, Step by Step)

Flexible docking in AutoDock Vina lets selected receptor side chains rotate while the backbone stays fixed. You split the receptor into a rigid PDBQT and a flex PDBQT with Meeko’s mk_prepare_receptor.py using the -f flag, then pass both to vina with --receptor and --flex. Backbone motion is not modelled.

This is the question that arrives about a week after your first successful rigid AutoDock Vina run. You look at your best pose, notice a lysine or an arginine sitting exactly where the ligand wants to be, and realise the protein in your PDB file was frozen in whatever conformation the crystallographer captured. This guide covers the flexible side chain workflow end to end, written by the StemSkills Lab team from 10+ years in structural bioinformatics, drug design and multiscale molecular modeling, including the part most tutorials skip: when flexible docking is worth the cost, and when rigid docking is the honest answer.

What does “flexible docking” actually mean in AutoDock Vina?

It means rotatable side chain torsions, and nothing else. The official Flexible docking tutorial opens by naming the problem directly: “The lack of receptor flexibility is arguably the greatest limitation in these types of docking methods. However, AutoDock Vina allows some limited flexibility of selected receptor side chains.”

Read the word “limited” carefully. What moves and what does not:

• Moves: the chi torsions of the residues you nominate, treated much like the rotatable bonds of a ligand.
• Does not move: the backbone. No loop rearrangement, no helix shift, no domain closure, no pocket opening.
• Does not move: every other side chain in the protein. Only the ones you list.

Mechanically, the receptor is split into two PDBQT files. The Meeko documentation on the flexible receptor PDBQT format describes the arrangement: atoms of each flexible residue sit between BEGIN_RES and END_RES records, the CA atom serves as the root, and “The atoms included in the flexible residue PDBQT must be omitted from the PDBQT for the rigid portions of the receptor.” That last sentence is the source of the most common silent failure in this workflow, and we come back to it in the troubleshooting section.

One number worth keeping in mind before you add torsions: Vina’s original benchmark, reported by Trott and Olson in AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading (J Comput Chem, 2010), was redocking of the 190 receptor-ligand complexes in the AutoDock 4 training set, with roughly two orders of magnitude improvement in speed. That benchmark flexed the ligand against a rigid receptor. The published accuracy figures you have read about Vina are rigid-receptor figures. Adding --flex takes you outside the regime the numbers were measured in.

When is flexible docking worth it, and when is rigid docking the honest choice?

Use flexible side chains when you have a specific, defensible reason to expect one or two named residues to reorganise. Structural evidence counts: the same residue adopting different rotamers across several PDB entries of the same protein, a mutagenesis result pointing at a gatekeeper residue, or a bound-versus-apo comparison where one side chain clearly swings. A general feeling that “proteins are flexible” is not a reason, because it does not tell you which residues to pick.

The honest default for a first project is rigid docking into a well-chosen structure. If a co-crystallised ligand of similar chemotype already exists, the pocket is already shaped correctly and flexible side chains buy you little. Flexible docking earns its cost most clearly in cross-docking, where you dock a ligand into a receptor that was solved with a different ligand, or none at all. Vina’s own FAQ lists this among the reasons a docking run fails: if you are not redocking, “perhaps the induced fit effects are large enough to affect the outcome of the docking experiment.”

Which residues should you make flexible, and how many?

Two to four. That is the practical answer for a student project, and the reason is combinatorial. Each added torsion enlarges the space the search algorithm has to cover, and Vina’s FAQ states that the number of steps in a run is determined heuristically “depending on the size and flexibility of the ligand and the flexible side chains.” Ten flexible residues do not give you a better answer. They give you a search problem your exhaustiveness setting cannot cover, and poses that look confident and are not.

Cost also depends on which residues you pick, because side chains differ in how many heavy-atom torsions they contribute:

Proline is a special case: its side chain closes back onto the backbone, so it is not a candidate for this treatment. Do not trust the table blindly either. The flex PDBQT written by your preparation tool states its own count in a REMARK line, for example “REMARK 2 active torsions” for a phenylalanine in the Meeko documentation example. Open the file and read it.

Two selection rules that save runs:

1. The residue must line the pocket you are docking into. Identify it properly with a pocket detection tool rather than by eye; our guide to finding a protein’s binding site for docking covers fpocket, CASTp and PrankWeb.
2. The residue must sit inside the grid box. The Vina FAQ is blunt about box size (“As small as possible, but not smaller”) and warns that the search “will not explore ligand and flexible side chain atom positions outside the search space.” A flexible residue half outside the box is a flexible residue that cannot move usefully. Set the box first, following the grid box centre and size procedure, then confirm your chosen residues fall inside it.

How do you split the receptor into rigid and flex PDBQT files?

With Meeko’s mk_prepare_receptor.py, which is the current preparation route in the official tutorials. The tutorial system is imatinib cross-docked into c-Abl from PDB entry 1FPU, treating Thr315 as flexible. Start from a receptor file that already carries all hydrogen atoms, then run:

mk_prepare_receptor.py -i 1fpu_receptorH.pdb -o 1fpu_receptor -p -v \
--box_size 20 20 20 --box_center 15.190 53.903 16.917 \
-f A:315 -a

Flag by flag, as the documentation defines them:

• -i input PDB, -o basename for the outputs.
• -p write the receptor PDBQT.
• -v write a box TXT file and a box PDB file, using --box_size and --box_center.
• -f A:315 the flexible residue, given as chain ID and residue number.
• -a ignore residues that are only partially resolved and fail to match the templates.

You get two files back:

1fpu_receptor_rigid.pdbqt           # rigid part
1fpu_receptor_flex.pdbqt            # flexible sidechain of Thr315

For more than one flexible residue, the Meeko options reference gives the syntax for -f/--flexres as chain ID and residue number pairs, comma separated inside quotes, for example -f ":42,B:23". An empty chain ID before the colon matches a file with no chain identifier, which is common in structures you have edited by hand.

There is an older route through AutoDockTools, still widely used in lab protocols: prepare_flexreceptor4.py, shipped with the Vina examples as prepare_flexreceptor.py. Its usage string specifies -r receptor_filename and -s for the flexible residue specification, where residue names are joined by _ (ARG8_ILE84) and the full form is written as molname:chainid:resname. This script is Python 2 code and must be run through MGLTools’ pythonsh. Calling it with a modern python3 raises a syntax error on the first print statement, which reads like a broken installation and is not one.

How do you run AutoDock Vina with –flex?

Pass the rigid file to --receptor and the flex file to --flex. Using the Vina scoring function, where the maps are computed internally:

vina --receptor 1fpu_receptor_rigid.pdbqt --flex 1fpu_receptor_flex.pdbqt \
     --ligand 1iep_ligand.pdbqt --config 1fpu_receptor.box.txt \
     --exhaustiveness 32 --out 1fpu_ligand_flex_vina_out.pdbqt

Using the AutoDock4 scoring function, affinity maps are precalculated with autogrid4 for the rigid part only, and there is no --receptor argument at all:

vina --flex 1fpu_receptor_flex.pdbqt --ligand 1iep_ligand.pdbqt \
     --maps 1fpu_receptor_rigid --scoring ad4 \
     --exhaustiveness 32 --out 1fpu_ligand_flex_ad4_out.pdbqt

Three run-time decisions matter. The tutorials use exhaustiveness 32 rather than the default 8, and with added torsions you should treat 32 as a floor. Keep the box unchanged from your rigid run if you intend to compare the two. And run the docking several times with different seeds: the search is stochastic, and a larger search space widens run-to-run variation. If three seeds give three different side chain conformations, that is your answer about how much to trust the result.

How do you read the output and check the side chain actually moved?

The output PDBQT holds both the ligand poses and the corresponding side chain conformations, which is what makes flexible docking output different from a rigid run. Two checks are worth doing every time.

Check that the side chain moved at all. Load the input receptor and the output pose in PyMOL or ChimeraX, superpose on the backbone, and measure chi1 for the flexible residue in both. If the torsion is unchanged, flexible docking found no reason to move it and your rigid result was already the right model. That is a valid finding, and reporting it beats presenting the flexible run as though it revealed something.

Check that the score difference is interpretable. This is where flexible docking gets misreported most often. Two rules, both from the official documentation:

• The tutorial carries an explicit warning: “Please don’t forget that energy scores giving by the AutoDock and Vina forcefield are not comparable between each other.” An ad4 score of -14.2 and a Vina score of -11.63 in the same tutorial describe the same system on two different scales.
• Scores from different receptor files are not comparable either. The basic tutorial redocks imatinib into its own structure (1IEP) and reports a best Vina score of -13.23 kcal/mol; the flexible tutorial cross-docks the same ligand into 1FPU with Thr315 flexible and reports -11.63 kcal/mol. Those two numbers do not say that flexibility made binding worse. They say the two runs used different receptors.

The comparison that is valid is rigid versus flexible on the same receptor file, same box, same scoring function, same exhaustiveness, several seeds each. That is a clean two-arm experiment, and it is the one to describe in your docking methods section. For turning the resulting poses into interpretable contacts, see our guide to interpreting molecular docking results.

Troubleshooting: what goes wrong with flexible docking, and how to fix it

Where does flexible docking sit in a full project?

It is a refinement step, not a starting point. A sensible order is: prepare the protein and ligand, define the pocket, run rigid docking, validate the protocol by redocking the native ligand and measuring RMSD against the crystal pose, and only then ask whether a specific side chain is blocking a result you have reason to expect. If the honest answer to “which residue, and why?” is a shrug, the next step is not --flex. It is either blind docking to check you have the right site, or a short protein-ligand MD simulation, which relaxes the whole pocket rather than the torsions you happened to nominate.

Flexible docking is one technique inside a larger skill set. Our molecular docking pillar guide maps how the pieces connect, and the computational biology skills roadmap shows where docking sits relative to structure preparation, MD and cheminformatics.

Frequently asked questions

Can AutoDock Vina move the protein backbone?

No. Vina’s flexible docking applies to selected side chain torsions only. Backbone motion, loop rearrangement and pocket opening are outside what --flex models. Use ensemble docking or molecular dynamics for those.

How many flexible residues can I use in AutoDock Vina?

There is no hard cap in the software, but two to four is the practical range. Every added torsion enlarges the search space, and Vina determines the number of search steps partly from the flexibility of the ligand and the flexible side chains, so cost rises quickly while reliability falls.

Do I need to increase exhaustiveness for flexible docking?

Yes. The official tutorials already use --exhaustiveness 32 instead of the default 8 for this system, and a larger search space needs at least that. Run several seeds and compare, rather than trusting one run.

Are flexible and rigid docking scores comparable?

Only when everything else is identical: same receptor file, same box, same scoring function. Scores from different scoring functions are not comparable, and neither are scores from different receptor structures.

Should I use Meeko or AutoDockTools to prepare flexible residues?

Meeko’s mk_prepare_receptor.py is the route used in the current AutoDock Vina tutorials and writes the rigid and flex PDBQT files together, which avoids the duplicated-atom problem. The AutoDockTools script still works and appears in many published protocols, but it runs under Python 2 through pythonsh.