Merge from tree-cleanup-branch: VRP, store CCP, store copy-prop, incremental SSA updating of FUD chains and newly exposed symbols. * Makefile.in (tree-ssa-copy.o): Depend on tree-ssa-propagate.h. (OBJS-common): Add tree-vrp.o. (tree-vrp.o): New rule. * basic-block.h (nearest_common_dominator_for_set): Declare. * common.opt (ftree-store-ccp): New flag. (ftree-copy-prop): New flag. (ftree-vrp): New flag. (ftree-store-copy-prop): New flag. * dominance.c (nearest_common_dominator_for_set): New. * domwalk.c (walk_dominator_tree): Only traverse statements in blocks marked in walk_data->interesting_blocks. * domwalk.h (struct dom_walk_data): Add field interesting_blocks. * fold-const.c (fold): Handle ASSERT_EXPR. * opts.c (decode_options): Set flag_tree_copy_prop at -O1. Set flag_tree_store_ccp, flag_tree_store_copy_prop and flag_tree_vrp at -O2. * timevar.def (TV_TREE_VRP): Define. (TV_TREE_COPY_PROP): Define. (TV_TREE_STORE_COPY_PROP): Define. (TV_TREE_SSA_INCREMENTAL): Define. (TV_TREE_STORE_CCP): Define. * tree-cfg.c (tree_can_merge_blocks_p): Remove reference to kill_redundant_phi_nodes from comment. (verify_expr): Handle ASSERT_EXPR. * tree-dfa.c (mark_new_vars_to_rename): Remove second argument. Update all users. (mark_call_clobbered_vars_to_rename): Remove. Update all users. * tree-flow-inline.h (unmodifiable_var_p): New. * tree-flow.h (enum value_range_type): Declare. (struct value_range_def): Declare. (value_range): Declare. (remove_all_phi_nodes_for): Remove. Update all users. (find_phi_node_for): Declare. (add_type_alias): Declare. (count_uses_and_derefs): Declare. (kill_redundant_phi_nodes): Remove. (rewrite_into_ssa): Remove. (rewrite_def_def_chains): Remove. (update_ssa, register_new_name_mapping, create_new_def_for, need_ssa_update_p, name_registered_for_update_p, release_ssa_name_after_update_ssa, dump_repl_tbl, debug_repl_tbl, dump_names_replaced_by, debug_names_replaced_by, mark_sym_for_renaming, mark_set_for_renaming, get_current_def, set_current_def, get_value_range, dump_value_range, debug_value_range, dump_all_value_ranges, debug_all_value_ranges, expr_computes_nonzero, loop_depth_of_name, unmodifiable_var_p): Declare. * tree-gimple.c (is_gimple_formal_tmp_rhs): Handle ASSERT_EXPR. * tree-into-ssa.c (block_defs_stack): Update comment. (old_ssa_names, new_ssa_names, old_virtual_ssa_names, syms_to_rename, names_to_release, repl_tbl, need_to_initialize_update_ssa_p, need_to_update_vops_p, need_to_replace_names_p): New locals. (NAME_SETS_GROWTH_FACTOR): Define. (struct repl_map_d): Declare. (struct mark_def_sites_global_data): Add field interesting_blocks. (enum rewrite_mode): Declare. (REGISTER_DEFS_IN_THIS_STMT): Define. (compute_global_livein): Use last_basic_block instead of n_basic_blocks. (set_def_block): Remove last argument. Update all callers. (prepare_use_operand_for_rename): Remove. Update all callers. (prepare_def_operand_for_rename): Remove. Update all callers. (symbol_marked_for_renaming): New. (is_old_name): New. (is_new_name): New. (repl_map_hash): New. (repl_map_eq): New. (repl_map_free): New. (names_replaced_by): New. (add_to_repl_tbl): New. (add_new_name_mapping): New. (mark_def_sites): Assume that all the operands in the statement are in normal form. (find_idf): Assert that the block in the stack is valid. (get_default_def_for): New. (insert_phi_nodes_for): Add new argument 'update_p'. Add documentation. If update_p is true, add a new mapping between the LHS of each new PHI and the name that it replaces. (insert_phi_nodes_1): Only call find_idf if needed. (get_reaching_def): Call get_default_def_for. (rewrite_operand): Remove. (rewrite_stmt): Do nothing if REGISTER_DEFS_IN_THIS_STMT and REWRITE_THIS_STMT are false. Assume that all the operands in the statement are in normal form. (rewrite_add_phi_arguments): Don't use PHI_REWRITTEN. (rewrite_virtual_phi_arguments): Remove. (invalidate_name_tags): Remove. (register_new_update_single, register_new_update_set, rewrite_update_init_block, replace_use, rewrite_update_fini_block, rewrite_update_stmt, rewrite_update_phi_arguments): New. rewrite_blocks): Remove argument 'fix_virtual_phis'. Add arguments 'entry', 'what' and 'blocks'. Initialize the dominator walker according to 'what' and 'blocks'. Start the dominator walk at 'entry'. (mark_def_site_blocks): Add argument 'interesting_blocks'. Use it to configure the dominator walker. (rewrite_into_ssa): Remove argument 'all'. Make internal. (rewrite_all_into_ssa): Remove. (rewrite_def_def_chains): Remove. (mark_def_interesting, mark_use_interesting, prepare_phi_args_for_update, prepare_block_for_update, prepare_def_site_for, prepare_def_sites, dump_names_replaced_by, debug_names_replaced_by, dump_repl_tbl, debug_repl_tbl, init_update_ssa, delete_update_ssa, create_new_def_for, register_new_name_mapping, mark_sym_for_renaming, mark_set_for_renaming, need_ssa_update_p, name_registered_for_update_p, ssa_names_to_replace, release_ssa_name_after_update_ssa, insert_updated_phi_nodes_for, update_ssa): New. * tree-loop-linear.c (linear_transform_loops): Call update_ssa instead of rewrite_into_ssa. * tree-optimize.c (vars_to_rename): Remove. Update all users. (init_tree_optimization_passes): Replace pass_redundant_phi with pass_copy_prop. Add pass_vrp. Replace pass_ccp with pass_store_ccp. Add pass_store_copy_prop after pass_store_ccp. (execute_todo): If the TODO_ flags don't include updating the SSA form, assert that it does not need to be updated. Call update_ssa instead of rewrite_into_ssa and rewrite_def_def_chains. If TODO_verify_loops is set, call verify_loop_closed_ssa. (tree_rest_of_compilation): * tree-pass.h (TODO_dump_func, TODO_ggc_collect, TODO_verify_ssa, TODO_verify_flow, TODO_verify_stmts, TODO_cleanup_cfg): Renumber. (TODO_verify_loops, TODO_update_ssa, TODO_update_ssa_no_phi, TODO_update_ssa_full_phi, TODO_update_ssa_only_virtuals): Define. (pass_copy_prop, pass_store_ccp, pass_store_copy_prop, pass_vrp): Declare. * tree-phinodes.c (make_phi_node): Update documentation. (remove_all_phi_nodes_for): Remove. (find_phi_node_for): New. * tree-pretty-print.c (dump_generic_node): Handle ASSERT_EXPR. * tree-scalar-evolution.c (follow_ssa_edge_in_rhs): Likewise. (interpret_rhs_modify_expr): Likewise. * tree-sra.c (decide_instantiations): Mark all symbols in SRA_CANDIDATES for renaming. (mark_all_v_defs_1): Rename from mark_all_v_defs. (mark_all_v_defs): New function. Update all users to call it with the whole list of scalarized statements, not just the first one. * tree-ssa-alias.c (count_ptr_derefs): Make extern. (compute_flow_insensitive_aliasing): If the tag is unmodifiable and the variable isn't or vice-versa, don't make them alias of each other. (setup_pointers_and_addressables): If the type tag for VAR is about to change, mark the old one for renaming. (add_type_alias): New. * tree-ssa-ccp.c: Document SSA-CCP and STORE-CCP. (ccp_lattice_t): Rename from latticevalue. (value): Remove. Update all users. (const_val): New local variable. (do_store_ccp): New local variable. (dump_lattice_value): Handle UNINITIALIZED. (debug_lattice_value): New. (get_default_value): Re-write. (set_lattice_value): Re-write. (def_to_varying): Remove. Update all users. (likely_value): Return VARYING for statements that make stores when STORE_CCP is false. Return VARYING for any statement other than MODIFY_EXPR, COND_EXPR and SWITCH_EXPR. (ccp_initialize): Re-write. (replace_uses_in, replace_vuse_in, substitute_and_fold): Move to tree-ssa-propagate.c. (ccp_lattice_meet): Handle memory stores when DO_STORE_CCP is true. (ccp_visit_phi_node): Likewise. (ccp_fold): Likewise. (evaluate_stmt): Likewise. (visit_assignment): Likewise. (ccp_visit_stmt): Likewise. (execute_ssa_ccp): Add argument 'store_ccp'. Copy it into DO_STORE_CCP. (do_ssa_ccp): New. (pass_ccp): Use it. (do_ssa_store_ccp): New. (gate_store_ccp): New. (pass_store_ccp): Declare. * tree-ssa-copy.c: Include tree-ssa-propagate.h. (may_propagate_copy): Reformat. Don't abort if ORIG is a virtual and DEST isn't. If NEW does not have alias information but DEST does, copy it. (copy_of, cached_last_copy_of, do_store_copy_prop, enum copy_prop_kind, which_copy_prop): Declare. (stmt_may_generate_copy, get_copy_of_val, get_last_copy_of, set_copy_of_val, dump_copy_of, copy_prop_visit_assignment, copy_prop_visit_cond_stmt, copy_prop_visit_stmt, copy_prop_visit_phi_node, init_copy_prop, fini_copy_prop, execute_copy_prop, gate_copy_prop, do_copy_prop, gate_store_copy_prop, store_copy_prop): New. (pass_copy_prop, pass_store_copy_prop): Declare. * tree-ssa-dom.c (struct opt_stats_d): Add fields 'num_const_prop' and 'num_copy_prop'. (cprop_operand): Update them. (dump_dominator_optimization_stats): Dump them. (tree_ssa_dominator_optimize): Call update_ssa instead of rewrite_into_ssa. (loop_depth_of_name): Declare extern. (simplify_cond_and_lookup_avail_expr): Guard against NULL values for LOW or HIGH. (cprop_into_successor_phis): Only propagate if NEW != ORIG. (record_equivalences_from_stmt): Call expr_computes_nonzero. (cprop_operand): Only propagate if VAL != OP. * tree-ssa-dse.c (dse_optimize_stmt): Mark symbols in removed statement for renaming. * tree-ssa-loop-im.c (move_computations): Call update_ssa. * tree-ssa-loop-ivopts.c (rewrite_address_base): Call add_type_alias if necessary. Call mark_new_vars_to_rename. (tree_ssa_iv_optimize): If new symbols need to be renamed, mark every statement updated, call update_ssa and rewrite_into_loop_closed_ssa. * tree-ssa-loop-manip.c (add_exit_phis): Do not remove DEF_BB from LIVEIN if VAR is a virtual. * tree-ssa-loop.c (tree_loop_optimizer_init): Call update_ssa. * tree-ssa-operands.c (get_expr_operands): Handle ASSERT_EXPR. (get_call_expr_operands): Reformat statement. (add_stmt_operand): Don't create V_MAY_DEFs for read-only symbols. * tree-ssa-propagate.c (ssa_prop_init): Initialize SSA_NAME_VALUE for every name. (first_vdef, stmt_makes_single_load, stmt_makes_single_store, get_value_loaded_by): New. (replace_uses_in, replace_vuses_in, replace_phi_args_in, substitute_and_fold): Move from tree-ssa-ccp.c. * tree-ssa-propagate.h (struct prop_value_d, prop_value_t, first_vdef, stmt_makes_single_load, stmt_makes_single_store, get_value_loaded_by, replace_uses_in, substitute_and_fold): Declare. * tree-ssa.c (verify_use): Fix error message. (propagate_into_addr, replace_immediate_uses, get_eq_name, check_phi_redundancy, kill_redundant_phi_nodes, pass_redundant_phi): Remove. Update all users. * tree-vect-transform.c (vect_create_data_ref_ptr): Call add_type_alias, if necessary. * tree-vectorizer.h (struct _stmt_vect_info): Update documentation for field 'memtag'. * tree-vrp.c: New file. * tree.def (ASSERT_EXPR): Define. * tree.h (ASSERT_EXPR_VAR): Define. (ASSERT_EXPR_COND): Define. (SSA_NAME_VALUE_RANGE): Define. (struct tree_ssa_name): Add field 'value_range'. (PHI_REWRITTEN): Remove. (struct tree_phi_node): Remove field 'rewritten'. * doc/invoke.texi (-fdump-tree-storeccp, -ftree-copy-prop, -ftree-store-copy-prop): Document. * doc/tree-ssa.texi: Remove broken link to McCAT's compiler. Document usage of update_ssa. testsuite/ChangeLog * g++.dg/tree-ssa/pr18178.C: New test. * gcc.c-torture/execute/20030216-1.x: Ignore at -O1. * gcc.c-torture/execute/20041019-1.c: New test. * gcc.dg/tree-ssa/20041008-1.c: New test. * gcc.dg/tree-ssa/ssa-ccp-12.c: New test. * gcc.dg/tree-ssa/20030731-2.c: Update to use -fdump-tree-store_ccp. * gcc.dg/tree-ssa/20030917-1.c: Likewise. * gcc.dg/tree-ssa/20030917-3.c: Likewise. * gcc.dg/tree-ssa/20040721-1.c: Likewise. * gcc.dg/tree-ssa/ssa-ccp-1.c: Likewise. * gcc.dg/tree-ssa/ssa-ccp-2.c: Likewise. * gcc.dg/tree-ssa/ssa-ccp-3.c: Likewise. * gcc.dg/tree-ssa/ssa-ccp-7.c: Likewise. * gcc.dg/tree-ssa/ssa-ccp-9.c: Likewise. From-SVN: r97884
683 lines
18 KiB
C
683 lines
18 KiB
C
/* High-level loop manipulation functions.
|
|
Copyright (C) 2004, 2005 Free Software Foundation, Inc.
|
|
|
|
This file is part of GCC.
|
|
|
|
GCC is free software; you can redistribute it and/or modify it
|
|
under the terms of the GNU General Public License as published by the
|
|
Free Software Foundation; either version 2, or (at your option) any
|
|
later version.
|
|
|
|
GCC is distributed in the hope that it will be useful, but WITHOUT
|
|
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
|
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
|
for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with GCC; see the file COPYING. If not, write to the Free
|
|
Software Foundation, 59 Temple Place - Suite 330, Boston, MA
|
|
02111-1307, USA. */
|
|
|
|
#include "config.h"
|
|
#include "system.h"
|
|
#include "coretypes.h"
|
|
#include "tm.h"
|
|
#include "tree.h"
|
|
#include "rtl.h"
|
|
#include "tm_p.h"
|
|
#include "hard-reg-set.h"
|
|
#include "basic-block.h"
|
|
#include "output.h"
|
|
#include "diagnostic.h"
|
|
#include "tree-flow.h"
|
|
#include "tree-dump.h"
|
|
#include "timevar.h"
|
|
#include "cfgloop.h"
|
|
#include "tree-pass.h"
|
|
#include "cfglayout.h"
|
|
#include "tree-scalar-evolution.h"
|
|
|
|
/* Creates an induction variable with value BASE + STEP * iteration in LOOP.
|
|
It is expected that neither BASE nor STEP are shared with other expressions
|
|
(unless the sharing rules allow this). Use VAR as a base var_decl for it
|
|
(if NULL, a new temporary will be created). The increment will occur at
|
|
INCR_POS (after it if AFTER is true, before it otherwise). INCR_POS and
|
|
AFTER can be computed using standard_iv_increment_position. The ssa versions
|
|
of the variable before and after increment will be stored in VAR_BEFORE and
|
|
VAR_AFTER (unless they are NULL). */
|
|
|
|
void
|
|
create_iv (tree base, tree step, tree var, struct loop *loop,
|
|
block_stmt_iterator *incr_pos, bool after,
|
|
tree *var_before, tree *var_after)
|
|
{
|
|
tree stmt, initial, step1, stmts;
|
|
tree vb, va;
|
|
enum tree_code incr_op = PLUS_EXPR;
|
|
|
|
if (!var)
|
|
{
|
|
var = create_tmp_var (TREE_TYPE (base), "ivtmp");
|
|
add_referenced_tmp_var (var);
|
|
}
|
|
|
|
vb = make_ssa_name (var, NULL_TREE);
|
|
if (var_before)
|
|
*var_before = vb;
|
|
va = make_ssa_name (var, NULL_TREE);
|
|
if (var_after)
|
|
*var_after = va;
|
|
|
|
/* For easier readability of the created code, produce MINUS_EXPRs
|
|
when suitable. */
|
|
if (TREE_CODE (step) == INTEGER_CST)
|
|
{
|
|
if (TYPE_UNSIGNED (TREE_TYPE (step)))
|
|
{
|
|
step1 = fold (build1 (NEGATE_EXPR, TREE_TYPE (step), step));
|
|
if (tree_int_cst_lt (step1, step))
|
|
{
|
|
incr_op = MINUS_EXPR;
|
|
step = step1;
|
|
}
|
|
}
|
|
else
|
|
{
|
|
if (!tree_expr_nonnegative_p (step)
|
|
&& may_negate_without_overflow_p (step))
|
|
{
|
|
incr_op = MINUS_EXPR;
|
|
step = fold (build1 (NEGATE_EXPR, TREE_TYPE (step), step));
|
|
}
|
|
}
|
|
}
|
|
|
|
stmt = build2 (MODIFY_EXPR, void_type_node, va,
|
|
build2 (incr_op, TREE_TYPE (base),
|
|
vb, step));
|
|
SSA_NAME_DEF_STMT (va) = stmt;
|
|
if (after)
|
|
bsi_insert_after (incr_pos, stmt, BSI_NEW_STMT);
|
|
else
|
|
bsi_insert_before (incr_pos, stmt, BSI_NEW_STMT);
|
|
|
|
initial = force_gimple_operand (base, &stmts, true, var);
|
|
if (stmts)
|
|
{
|
|
edge pe = loop_preheader_edge (loop);
|
|
|
|
bsi_insert_on_edge_immediate_loop (pe, stmts);
|
|
}
|
|
|
|
stmt = create_phi_node (vb, loop->header);
|
|
SSA_NAME_DEF_STMT (vb) = stmt;
|
|
add_phi_arg (stmt, initial, loop_preheader_edge (loop));
|
|
add_phi_arg (stmt, va, loop_latch_edge (loop));
|
|
}
|
|
|
|
/* Add exit phis for the USE on EXIT. */
|
|
|
|
static void
|
|
add_exit_phis_edge (basic_block exit, tree use)
|
|
{
|
|
tree phi, def_stmt = SSA_NAME_DEF_STMT (use);
|
|
basic_block def_bb = bb_for_stmt (def_stmt);
|
|
struct loop *def_loop;
|
|
edge e;
|
|
edge_iterator ei;
|
|
|
|
/* Check that some of the edges entering the EXIT block exits a loop in
|
|
that USE is defined. */
|
|
FOR_EACH_EDGE (e, ei, exit->preds)
|
|
{
|
|
def_loop = find_common_loop (def_bb->loop_father, e->src->loop_father);
|
|
if (!flow_bb_inside_loop_p (def_loop, e->dest))
|
|
break;
|
|
}
|
|
|
|
if (!e)
|
|
return;
|
|
|
|
phi = create_phi_node (use, exit);
|
|
|
|
FOR_EACH_EDGE (e, ei, exit->preds)
|
|
add_phi_arg (phi, use, e);
|
|
|
|
SSA_NAME_DEF_STMT (use) = def_stmt;
|
|
}
|
|
|
|
/* Add exit phis for VAR that is used in LIVEIN.
|
|
Exits of the loops are stored in EXITS. */
|
|
|
|
static void
|
|
add_exit_phis_var (tree var, bitmap livein, bitmap exits)
|
|
{
|
|
bitmap def;
|
|
unsigned index;
|
|
basic_block def_bb = bb_for_stmt (SSA_NAME_DEF_STMT (var));
|
|
bitmap_iterator bi;
|
|
|
|
if (is_gimple_reg (var))
|
|
bitmap_clear_bit (livein, def_bb->index);
|
|
else
|
|
bitmap_set_bit (livein, def_bb->index);
|
|
|
|
def = BITMAP_ALLOC (NULL);
|
|
bitmap_set_bit (def, def_bb->index);
|
|
compute_global_livein (livein, def);
|
|
BITMAP_FREE (def);
|
|
|
|
EXECUTE_IF_AND_IN_BITMAP (exits, livein, 0, index, bi)
|
|
{
|
|
add_exit_phis_edge (BASIC_BLOCK (index), var);
|
|
}
|
|
}
|
|
|
|
/* Add exit phis for the names marked in NAMES_TO_RENAME.
|
|
Exits of the loops are stored in EXITS. Sets of blocks where the ssa
|
|
names are used are stored in USE_BLOCKS. */
|
|
|
|
static void
|
|
add_exit_phis (bitmap names_to_rename, bitmap *use_blocks, bitmap loop_exits)
|
|
{
|
|
unsigned i;
|
|
bitmap_iterator bi;
|
|
|
|
EXECUTE_IF_SET_IN_BITMAP (names_to_rename, 0, i, bi)
|
|
{
|
|
add_exit_phis_var (ssa_name (i), use_blocks[i], loop_exits);
|
|
}
|
|
}
|
|
|
|
/* Returns a bitmap of all loop exit edge targets. */
|
|
|
|
static bitmap
|
|
get_loops_exits (void)
|
|
{
|
|
bitmap exits = BITMAP_ALLOC (NULL);
|
|
basic_block bb;
|
|
edge e;
|
|
edge_iterator ei;
|
|
|
|
FOR_EACH_BB (bb)
|
|
{
|
|
FOR_EACH_EDGE (e, ei, bb->preds)
|
|
if (e->src != ENTRY_BLOCK_PTR
|
|
&& !flow_bb_inside_loop_p (e->src->loop_father, bb))
|
|
{
|
|
bitmap_set_bit (exits, bb->index);
|
|
break;
|
|
}
|
|
}
|
|
|
|
return exits;
|
|
}
|
|
|
|
/* For USE in BB, if it is used outside of the loop it is defined in,
|
|
mark it for rewrite. Record basic block BB where it is used
|
|
to USE_BLOCKS. */
|
|
|
|
static void
|
|
find_uses_to_rename_use (basic_block bb, tree use, bitmap *use_blocks)
|
|
{
|
|
unsigned ver;
|
|
basic_block def_bb;
|
|
struct loop *def_loop;
|
|
|
|
if (TREE_CODE (use) != SSA_NAME)
|
|
return;
|
|
|
|
ver = SSA_NAME_VERSION (use);
|
|
def_bb = bb_for_stmt (SSA_NAME_DEF_STMT (use));
|
|
if (!def_bb)
|
|
return;
|
|
def_loop = def_bb->loop_father;
|
|
|
|
/* If the definition is not inside loop, it is not interesting. */
|
|
if (!def_loop->outer)
|
|
return;
|
|
|
|
if (!use_blocks[ver])
|
|
use_blocks[ver] = BITMAP_ALLOC (NULL);
|
|
bitmap_set_bit (use_blocks[ver], bb->index);
|
|
|
|
if (!flow_bb_inside_loop_p (def_loop, bb))
|
|
mark_for_rewrite (use);
|
|
}
|
|
|
|
/* For uses in STMT, mark names that are used outside of the loop they are
|
|
defined to rewrite. Record the set of blocks in that the ssa
|
|
names are defined to USE_BLOCKS. */
|
|
|
|
static void
|
|
find_uses_to_rename_stmt (tree stmt, bitmap *use_blocks)
|
|
{
|
|
ssa_op_iter iter;
|
|
tree var;
|
|
basic_block bb = bb_for_stmt (stmt);
|
|
|
|
get_stmt_operands (stmt);
|
|
|
|
FOR_EACH_SSA_TREE_OPERAND (var, stmt, iter, SSA_OP_ALL_USES | SSA_OP_ALL_KILLS)
|
|
find_uses_to_rename_use (bb, var, use_blocks);
|
|
}
|
|
|
|
/* Marks names that are used in BB and outside of the loop they are
|
|
defined in for rewrite. Records the set of blocks in that the ssa
|
|
names are defined to USE_BLOCKS. */
|
|
|
|
static void
|
|
find_uses_to_rename_bb (basic_block bb, bitmap *use_blocks)
|
|
{
|
|
block_stmt_iterator bsi;
|
|
edge e;
|
|
edge_iterator ei;
|
|
tree phi;
|
|
|
|
FOR_EACH_EDGE (e, ei, bb->succs)
|
|
for (phi = phi_nodes (e->dest); phi; phi = PHI_CHAIN (phi))
|
|
find_uses_to_rename_use (bb, PHI_ARG_DEF_FROM_EDGE (phi, e),
|
|
use_blocks);
|
|
|
|
for (bsi = bsi_start (bb); !bsi_end_p (bsi); bsi_next (&bsi))
|
|
find_uses_to_rename_stmt (bsi_stmt (bsi), use_blocks);
|
|
}
|
|
|
|
/* Marks names that are used outside of the loop they are defined in
|
|
for rewrite. Records the set of blocks in that the ssa
|
|
names are defined to USE_BLOCKS. If CHANGED_BBS is not NULL,
|
|
scan only blocks in this set. */
|
|
|
|
static void
|
|
find_uses_to_rename (bitmap changed_bbs, bitmap *use_blocks)
|
|
{
|
|
basic_block bb;
|
|
unsigned index;
|
|
bitmap_iterator bi;
|
|
|
|
if (changed_bbs)
|
|
{
|
|
EXECUTE_IF_SET_IN_BITMAP (changed_bbs, 0, index, bi)
|
|
{
|
|
find_uses_to_rename_bb (BASIC_BLOCK (index), use_blocks);
|
|
}
|
|
}
|
|
else
|
|
{
|
|
FOR_EACH_BB (bb)
|
|
{
|
|
find_uses_to_rename_bb (bb, use_blocks);
|
|
}
|
|
}
|
|
}
|
|
|
|
/* Rewrites the program into a loop closed ssa form -- i.e. inserts extra
|
|
phi nodes to ensure that no variable is used outside the loop it is
|
|
defined in.
|
|
|
|
This strengthening of the basic ssa form has several advantages:
|
|
|
|
1) Updating it during unrolling/peeling/versioning is trivial, since
|
|
we do not need to care about the uses outside of the loop.
|
|
2) The behavior of all uses of an induction variable is the same.
|
|
Without this, you need to distinguish the case when the variable
|
|
is used outside of the loop it is defined in, for example
|
|
|
|
for (i = 0; i < 100; i++)
|
|
{
|
|
for (j = 0; j < 100; j++)
|
|
{
|
|
k = i + j;
|
|
use1 (k);
|
|
}
|
|
use2 (k);
|
|
}
|
|
|
|
Looking from the outer loop with the normal SSA form, the first use of k
|
|
is not well-behaved, while the second one is an induction variable with
|
|
base 99 and step 1.
|
|
|
|
If CHANGED_BBS is not NULL, we look for uses outside loops only in
|
|
the basic blocks in this set. */
|
|
|
|
void
|
|
rewrite_into_loop_closed_ssa (bitmap changed_bbs)
|
|
{
|
|
bitmap loop_exits = get_loops_exits ();
|
|
bitmap *use_blocks;
|
|
unsigned i;
|
|
bitmap names_to_rename;
|
|
|
|
gcc_assert (!any_marked_for_rewrite_p ());
|
|
|
|
use_blocks = xcalloc (num_ssa_names, sizeof (bitmap));
|
|
|
|
/* Find the uses outside loops. */
|
|
find_uses_to_rename (changed_bbs, use_blocks);
|
|
|
|
if (!any_marked_for_rewrite_p ())
|
|
{
|
|
free (use_blocks);
|
|
BITMAP_FREE (loop_exits);
|
|
return;
|
|
}
|
|
|
|
/* Add the phi nodes on exits of the loops for the names we need to
|
|
rewrite. */
|
|
names_to_rename = marked_ssa_names ();
|
|
add_exit_phis (names_to_rename, use_blocks, loop_exits);
|
|
|
|
for (i = 0; i < num_ssa_names; i++)
|
|
BITMAP_FREE (use_blocks[i]);
|
|
free (use_blocks);
|
|
BITMAP_FREE (loop_exits);
|
|
BITMAP_FREE (names_to_rename);
|
|
|
|
/* Do the rewriting. */
|
|
rewrite_ssa_into_ssa ();
|
|
}
|
|
|
|
/* Check invariants of the loop closed ssa form for the USE in BB. */
|
|
|
|
static void
|
|
check_loop_closed_ssa_use (basic_block bb, tree use)
|
|
{
|
|
tree def;
|
|
basic_block def_bb;
|
|
|
|
if (TREE_CODE (use) != SSA_NAME)
|
|
return;
|
|
|
|
def = SSA_NAME_DEF_STMT (use);
|
|
def_bb = bb_for_stmt (def);
|
|
gcc_assert (!def_bb
|
|
|| flow_bb_inside_loop_p (def_bb->loop_father, bb));
|
|
}
|
|
|
|
/* Checks invariants of loop closed ssa form in statement STMT in BB. */
|
|
|
|
static void
|
|
check_loop_closed_ssa_stmt (basic_block bb, tree stmt)
|
|
{
|
|
ssa_op_iter iter;
|
|
tree var;
|
|
|
|
get_stmt_operands (stmt);
|
|
|
|
FOR_EACH_SSA_TREE_OPERAND (var, stmt, iter, SSA_OP_ALL_USES)
|
|
check_loop_closed_ssa_use (bb, var);
|
|
}
|
|
|
|
/* Checks that invariants of the loop closed ssa form are preserved. */
|
|
|
|
void
|
|
verify_loop_closed_ssa (void)
|
|
{
|
|
basic_block bb;
|
|
block_stmt_iterator bsi;
|
|
tree phi;
|
|
unsigned i;
|
|
|
|
verify_ssa (false);
|
|
|
|
FOR_EACH_BB (bb)
|
|
{
|
|
for (phi = phi_nodes (bb); phi; phi = PHI_CHAIN (phi))
|
|
for (i = 0; i < (unsigned) PHI_NUM_ARGS (phi); i++)
|
|
check_loop_closed_ssa_use (PHI_ARG_EDGE (phi, i)->src,
|
|
PHI_ARG_DEF (phi, i));
|
|
|
|
for (bsi = bsi_start (bb); !bsi_end_p (bsi); bsi_next (&bsi))
|
|
check_loop_closed_ssa_stmt (bb, bsi_stmt (bsi));
|
|
}
|
|
}
|
|
|
|
/* Split loop exit edge EXIT. The things are a bit complicated by a need to
|
|
preserve the loop closed ssa form. */
|
|
|
|
void
|
|
split_loop_exit_edge (edge exit)
|
|
{
|
|
basic_block dest = exit->dest;
|
|
basic_block bb = loop_split_edge_with (exit, NULL);
|
|
tree phi, new_phi, new_name, name;
|
|
use_operand_p op_p;
|
|
|
|
for (phi = phi_nodes (dest); phi; phi = PHI_CHAIN (phi))
|
|
{
|
|
op_p = PHI_ARG_DEF_PTR_FROM_EDGE (phi, single_succ_edge (bb));
|
|
|
|
name = USE_FROM_PTR (op_p);
|
|
|
|
/* If the argument of the phi node is a constant, we do not need
|
|
to keep it inside loop. */
|
|
if (TREE_CODE (name) != SSA_NAME)
|
|
continue;
|
|
|
|
/* Otherwise create an auxiliary phi node that will copy the value
|
|
of the ssa name out of the loop. */
|
|
new_name = duplicate_ssa_name (name, NULL);
|
|
new_phi = create_phi_node (new_name, bb);
|
|
SSA_NAME_DEF_STMT (new_name) = new_phi;
|
|
add_phi_arg (new_phi, name, exit);
|
|
SET_USE (op_p, new_name);
|
|
}
|
|
}
|
|
|
|
/* Insert statement STMT to the edge E and update the loop structures.
|
|
Returns the newly created block (if any). */
|
|
|
|
basic_block
|
|
bsi_insert_on_edge_immediate_loop (edge e, tree stmt)
|
|
{
|
|
basic_block src, dest, new_bb;
|
|
struct loop *loop_c;
|
|
|
|
src = e->src;
|
|
dest = e->dest;
|
|
|
|
loop_c = find_common_loop (src->loop_father, dest->loop_father);
|
|
|
|
new_bb = bsi_insert_on_edge_immediate (e, stmt);
|
|
|
|
if (!new_bb)
|
|
return NULL;
|
|
|
|
add_bb_to_loop (new_bb, loop_c);
|
|
if (dest->loop_father->latch == src)
|
|
dest->loop_father->latch = new_bb;
|
|
|
|
return new_bb;
|
|
}
|
|
|
|
/* Returns the basic block in that statements should be emitted for induction
|
|
variables incremented at the end of the LOOP. */
|
|
|
|
basic_block
|
|
ip_end_pos (struct loop *loop)
|
|
{
|
|
return loop->latch;
|
|
}
|
|
|
|
/* Returns the basic block in that statements should be emitted for induction
|
|
variables incremented just before exit condition of a LOOP. */
|
|
|
|
basic_block
|
|
ip_normal_pos (struct loop *loop)
|
|
{
|
|
tree last;
|
|
basic_block bb;
|
|
edge exit;
|
|
|
|
if (!single_pred_p (loop->latch))
|
|
return NULL;
|
|
|
|
bb = single_pred (loop->latch);
|
|
last = last_stmt (bb);
|
|
if (TREE_CODE (last) != COND_EXPR)
|
|
return NULL;
|
|
|
|
exit = EDGE_SUCC (bb, 0);
|
|
if (exit->dest == loop->latch)
|
|
exit = EDGE_SUCC (bb, 1);
|
|
|
|
if (flow_bb_inside_loop_p (loop, exit->dest))
|
|
return NULL;
|
|
|
|
return bb;
|
|
}
|
|
|
|
/* Stores the standard position for induction variable increment in LOOP
|
|
(just before the exit condition if it is available and latch block is empty,
|
|
end of the latch block otherwise) to BSI. INSERT_AFTER is set to true if
|
|
the increment should be inserted after *BSI. */
|
|
|
|
void
|
|
standard_iv_increment_position (struct loop *loop, block_stmt_iterator *bsi,
|
|
bool *insert_after)
|
|
{
|
|
basic_block bb = ip_normal_pos (loop), latch = ip_end_pos (loop);
|
|
tree last = last_stmt (latch);
|
|
|
|
if (!bb
|
|
|| (last && TREE_CODE (last) != LABEL_EXPR))
|
|
{
|
|
*bsi = bsi_last (latch);
|
|
*insert_after = true;
|
|
}
|
|
else
|
|
{
|
|
*bsi = bsi_last (bb);
|
|
*insert_after = false;
|
|
}
|
|
}
|
|
|
|
/* Copies phi node arguments for duplicated blocks. The index of the first
|
|
duplicated block is FIRST_NEW_BLOCK. */
|
|
|
|
static void
|
|
copy_phi_node_args (unsigned first_new_block)
|
|
{
|
|
unsigned i;
|
|
|
|
for (i = first_new_block; i < (unsigned) last_basic_block; i++)
|
|
BASIC_BLOCK (i)->rbi->duplicated = 1;
|
|
|
|
for (i = first_new_block; i < (unsigned) last_basic_block; i++)
|
|
add_phi_args_after_copy_bb (BASIC_BLOCK (i));
|
|
|
|
for (i = first_new_block; i < (unsigned) last_basic_block; i++)
|
|
BASIC_BLOCK (i)->rbi->duplicated = 0;
|
|
}
|
|
|
|
/* Renames variables in the area copied by tree_duplicate_loop_to_header_edge.
|
|
FIRST_NEW_BLOCK is the first block in the copied area. DEFINITIONS is
|
|
a bitmap of all ssa names defined inside the loop. */
|
|
|
|
static void
|
|
rename_variables (unsigned first_new_block, bitmap definitions)
|
|
{
|
|
unsigned i, copy_number = 0;
|
|
basic_block bb;
|
|
htab_t ssa_name_map = NULL;
|
|
|
|
for (i = first_new_block; i < (unsigned) last_basic_block; i++)
|
|
{
|
|
bb = BASIC_BLOCK (i);
|
|
|
|
/* We assume that first come all blocks from the first copy, then all
|
|
blocks from the second copy, etc. */
|
|
if (copy_number != (unsigned) bb->rbi->copy_number)
|
|
{
|
|
allocate_ssa_names (definitions, &ssa_name_map);
|
|
copy_number = bb->rbi->copy_number;
|
|
}
|
|
|
|
rewrite_to_new_ssa_names_bb (bb, ssa_name_map);
|
|
}
|
|
|
|
htab_delete (ssa_name_map);
|
|
}
|
|
|
|
/* Sets SSA_NAME_DEF_STMT for results of all phi nodes in BB. */
|
|
|
|
static void
|
|
set_phi_def_stmts (basic_block bb)
|
|
{
|
|
tree phi;
|
|
|
|
for (phi = phi_nodes (bb); phi; phi = PHI_CHAIN (phi))
|
|
SSA_NAME_DEF_STMT (PHI_RESULT (phi)) = phi;
|
|
}
|
|
|
|
/* The same as cfgloopmanip.c:duplicate_loop_to_header_edge, but also updates
|
|
ssa. In order to achieve this, only loops whose exits all lead to the same
|
|
location are handled.
|
|
|
|
FIXME: we create some degenerate phi nodes that could be avoided by copy
|
|
propagating them instead. Unfortunately this is not completely
|
|
straightforward due to problems with constant folding. */
|
|
|
|
bool
|
|
tree_duplicate_loop_to_header_edge (struct loop *loop, edge e,
|
|
struct loops *loops,
|
|
unsigned int ndupl, sbitmap wont_exit,
|
|
edge orig, edge *to_remove,
|
|
unsigned int *n_to_remove, int flags)
|
|
{
|
|
unsigned first_new_block;
|
|
basic_block bb;
|
|
unsigned i;
|
|
bitmap definitions;
|
|
|
|
if (!(loops->state & LOOPS_HAVE_SIMPLE_LATCHES))
|
|
return false;
|
|
if (!(loops->state & LOOPS_HAVE_PREHEADERS))
|
|
return false;
|
|
|
|
#ifdef ENABLE_CHECKING
|
|
verify_loop_closed_ssa ();
|
|
#endif
|
|
|
|
gcc_assert (!any_marked_for_rewrite_p ());
|
|
|
|
first_new_block = last_basic_block;
|
|
if (!duplicate_loop_to_header_edge (loop, e, loops, ndupl, wont_exit,
|
|
orig, to_remove, n_to_remove, flags))
|
|
return false;
|
|
|
|
/* Readd the removed phi args for e. */
|
|
flush_pending_stmts (e);
|
|
|
|
/* Copy the phi node arguments. */
|
|
copy_phi_node_args (first_new_block);
|
|
|
|
/* Rename the variables. */
|
|
definitions = marked_ssa_names ();
|
|
rename_variables (first_new_block, definitions);
|
|
unmark_all_for_rewrite ();
|
|
BITMAP_FREE (definitions);
|
|
|
|
/* For some time we have the identical ssa names as results in multiple phi
|
|
nodes. When phi node is resized, it sets SSA_NAME_DEF_STMT of its result
|
|
to the new copy. This means that we cannot easily ensure that the ssa
|
|
names defined in those phis are pointing to the right one -- so just
|
|
recompute SSA_NAME_DEF_STMT for them. */
|
|
|
|
for (i = first_new_block; i < (unsigned) last_basic_block; i++)
|
|
{
|
|
bb = BASIC_BLOCK (i);
|
|
set_phi_def_stmts (bb);
|
|
if (bb->rbi->copy_number == 1)
|
|
set_phi_def_stmts (bb->rbi->original);
|
|
}
|
|
|
|
scev_reset ();
|
|
#ifdef ENABLE_CHECKING
|
|
verify_loop_closed_ssa ();
|
|
#endif
|
|
|
|
return true;
|
|
}
|
|
|